Archive | User Experience

Accessibility Testing Tools: The Sometimes Unreliable Friends

When I started to do accessibility testing, I would rely too heavily on HTML validators to verify if a site is compliant. I would use the web developer toolbar, Accessibility-Testing-ToolsW3C validator, and would do some basic testing with a screen reader, not fully understanding the complexity of the needs of the end users.

As I gained more knowledge of the end users and the constraints of technology, I had to take a step back from reliance on the tools due to the many false negatives, and test the errors manually to ensure I reported actual bugs. For instance, the W3C validator would claim a variable had been duplicated, but one “duplicate” would be in a comment describing the variable — not a bug.

I have also found that what is marked as a semantic error is not an issue for assistive technology, but may be an issue if semantically correct (like an aria-label and a title for an href will have both read out). There are also some HTML validators, like the WAVE tool that will highlight errors that other validators will not highlight – so which one is correct?

Some other tools, like color ratio analyzers, will not highlight valid bugs, or mark a failure that is not actually incorrect. For instance, they will sometimes mark two colors as breaking the 1.4.3 checkpoint, but when you check the colors used by inspecting the elements, you find they do not actually break the checkpoint. One such example is the Juicy Studio tool — the Firefox add-on tends to have the result slightly out from the actual ratio compared to their website analyzer. Often, I will note the highlighted error, and then validate the actual colors used on the website to see if a failure is correct (which is also mentioned in a review for the tool).

Continue Reading →

Continue Reading

Resources for Learning About Usability Testing

I recently stumbled upon the website and was surprised to learn that the site is managed by the Digital Communications Division in the U.S. Department of Health and Human Services Office.

The site offers a wealth of resources to learn more about user experience (UX) best practices and other UX disciplines like Information Architecture (IA), User Interface Design, Interaction Design (IxD), and Accessibility. This user-centered design process map is great step-by-step guide to website development from this perspective. As the page notes, “the type of site you are developing, your requirementsteam, timeline, and the environment in which you are developing will determine the tasks you perform and the order in which you perform them.”

From a testing perspective, uTest University offers several courses to help you get started with Usability Testing beginning with The Basics of Usability Testing for Desktop and Web Apps. This course covers the basics of usability testing, including what it is, why it’s needed, and basic implementation and execution details. In an Introduction to uTest Usability Testing, you will gain an understanding of how to participate in Usability test cycles when you are invited to them.

The course includes these four tips from a usability expert:

Continue Reading →

Continue Reading

Accessibility Testing: The Answer to the Common Question of ‘Why?’

Often when I recommend to project managers or developers that a project should include accessibility testing, I w3c-1299614815will get asked, “Why?” Not always a simple ‘why,’ but a “Why would a blind person use this site?”; “Why is a screen reader important on a mobile?”; “Why would anyone use just a keyboard?”; “Why are color ratios important?”

Often, all these questions can be chalked up to just not understanding the audience requiring an accessible website. I will try to educate as best as possible by giving some statistics and reasons as to why accessibility is important.

First, the testing is not just covering blind users, but those that are deaf, short-sighted, arthritic, color-blind, dyslexic – the list is actually quite exhaustive. Screen reader users, for instance, are not just users with a sight issue, but sometimes people who find it easier to have the content read out to them, like someone who finds the screen too small or has issues that make reading more difficult.

Another point many business users will take an interest in is the potential loss of business by not being accessible, and/or potential legal cases. I have given a few examples of this in uTest University tutorials on accessibility, so please check them out for more details.

I guess that with all types of testing, we all get the same questions around why it is important, but I am surprised when I get asked this by testers I work with. Is accessibility lesser to other versions of testing? Sometimes the same testers will assume that accessibility testing is either easy or subjective and not a true form of reproducible testing. I have found this can be the case, but not necessarily.

Continue Reading →

Continue Reading

Testing the Limits with Craig Tomlin, Usability Expert: Part II

In the second part of this two-part interview, Usability Expert Craig Tomlin talks the best user experiences and the future of usabCraig Tomlinility. Be sure to follow Craig on Twitter @ctomlin, and get to know him along with the first part of our interview.

uTest: When functional testing, you’ll probably start with the mission-critical functionality. When an end-user is given an app or site to provide Usability feedback without direction, which areas are most critical to look at first?

CT: Usability feedback without direction is just opinion, which is not valuable. Usability testing is always conducted on critical tasks. What are critical tasks? Those are the tasks that must be accomplished for the user to be successful. Note that it’s the ‘user successful’ part of that definition that is so important. Many times, businesses seek to optimize tasks that are important for their business, but not at all important to the user. That’s a waste of resources. Yes, businesses must be successful, but ultimately that success only comes from providing a valuable service or product to the target audience. Making sure the user experience is maximized for the user is the best way for a business to be successful.

So usability should never be based on feedback with no direction. Usability if done properly identifies critical tasks, the personas of representative users who need to accomplish that task, a protocol that tests the task in a non-biased manner, and the metrics that will provide the insight into whether that task is being accomplished as efficiently and effectively as possible, and with the best possible satisfaction.

uTest: Give us your definition of ‘the best user experience.’

CT: First, it’s important to define what we mean when we say ‘user experience.’ I’ve been known to rant about this subject, because it’s rare to find two people that have the exact same definition of what ‘user experience’ actually means. The original concept that was popularized by Don Norman was a broad or holistic viewpoint on how design and humans connect, and incorporated much more than just a UI or specific functionality.

In that broad context, the user experience you have with a brand includes your experiences with the brand’s product, with their education, marketing and sales communications, with their customer service be it in store, online or via phone, with the pleasure and satisfaction you receive from using the product or service, and with your interaction with others involved with usage of that product or service.

If we use that definition of user experience, then the best user experience is something that was designed with our needs in mind to give us satisfaction and pleasure from using it again and again. Consider an iPhone or iPad, or even an exit door that opens automatically when you approach it. All are examples of the best user experience.

I think it’s sometimes easy for us to forget that definition, to become so wrapped up in our own unique smaller piece of the broader user experience, that we miss opportunities to truly make a best user experience for our customers and clients. Taking a step back and remembering the bigger, broader definition of user experience can help to reinforce how what we do is making a difference for the people that use our products or services.

Continue Reading →

Continue Reading

Testing the Limits with Craig Tomlin, Usability Expert: Part I

Our guest in this installment of Testing the Limits is Craig Tomlin, an award-winning digital marketing and User Experience (UX) consultant with oCraig Tomlinver 20 years’ experience in B2B and B2C demand generation and eCommerce. He leads marketing and UX strategies and tactics for firms like: Blue Cross Blue Shield, BMC Software, Disney, IBM, Kodak, Prudential, WellPoint and more. You can follow Craig on Twitter @ctomlin

In the first part of this two-part interview, Craig talks the differences between Usability and functional testing, and why Usability often doesn’t get the spotlight it deserves. 

uTest: What initially drew you into exploring usability?

CT: I was drawn to usability in the mid-1990s, when preparing to redesign 22 websites for WellPoint Health Networks, a major health insurance corporation in the U.S. My earlier attempts at redesigning sites sometimes failed miserably. I didn’t understand why. After all, we had brilliant I.T. teams, the best website design vendors, internal stakeholders that knew their products and subject matter backwards and forwards. So why the failures? I realized the missing ingredient was the users.

Without user-centered design, we were just guessing about important usability issues when designing new experiences. I took it upon myself to become certified in usability and apply user-centered design principles to the 22 website redesign project. We conducted extensive usability testing, and created mental maps, information architectures and taxonomies based on user-defined requirements. Because of the inclusion of users as part of the process, we were very successful with our new designs. I’ve been a big fan of usability and user-centered design ever since.

uTest: When folks think of usability, it can often get muddied or confused with the functionality of a site. What goes into a Usability Audit?

CT: The International Organization for Standardization (ISO) defines usability as “The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.” Therefore, usability includes both functionality (the effectiveness and efficiency portion) and satisfaction, of which both can be measured using quantifiable data. Usability testing if done properly includes evaluations of the critical tasks associated with a website or app in conjunction with the overall satisfaction of that experience.

As to a usability audit, it is an evaluation of effectiveness, efficiency and satisfaction using a well-defined set of criteria that can be reproduced over any number of websites, apps or other objects. I think that strictly speaking, functional testing only evaluates that effectiveness portion of the definition, in that a function either works as specified, or does not work as specified. Any data on efficiency and satisfaction would typically not be part of a functional test.

uTest: You’re one of 4,900 certified usability analysts in the world. How does one get certified, and what differentiates a certified usability analyst from an ordinary tester who’s providing their off-the-cuff feedback about a site or app?

CT: Becoming a certified usability analyst was very beneficial to me, my clients and the people who use the websites and applications I create. It’s interesting that accountants, doctors, lawyers and even hair stylists all have to become qualified through passing exams before they start dealing with real clients. Isn’t it sad that anyone can call themselves a ‘usability expert’ even though they have absolutely no training at all in usability? It’s a real case of ‘buyer beware’ for anyone seeking a knowledgeable usability expert to help them improve their website or app user experience. I’ve seen plenty of examples of untrained usability advisors providing bad advice to clients that caused them harm.

Being certified means that person has taken the time to learn the appropriate skills to properly test usability principles and a mark of someone who not only talks the talk, but also walks the walk. There are multiple ways to become educated in usability, including taking usability and human computer interaction courses at universities, or doing what I did and becoming certified through the Human Factors International’s CUA program. The 4,900 or so CUAs in the HFI program all completed an extensive set of educational programs and then passed the test (it took me two and a half hours to complete). I highly recommend either going through a university program or taking the CUA course to become a proficient usability analyst.

Continue Reading →

Continue Reading

Understanding Accessibility Testing

uTest UniversityAs we live more of our lives online – everything from booking travel and shopping, to bill payment, watching TV shows or movies, and reading the news – a greater emphasis is put on the ability to access web sites and mobile apps, especially for people with disabilities. Of the 241.7 million adults aged 15 and older, 6.2% experience some level of difficulty with seeing, hearing, or having their speech understood, according to the 2010 U.S.Census Bureau Americans with Disabilities report.

Take into account the growing segment of adults who suffer from age-related sight or hearing loss and you begin to see a significant population who can be inadvertently shut out from using sites or apps that are not accessible. Companies who don’t pay attention to accessibility standards risk losing revenue from lost reservations or communications, and may even face litigation.

Web accessibility was brought to the forefront by a landmark class action lawsuit filed by the National Federation of the Blind (NFB) against Target Corporation over Target settled with the NFB for $6 million in 2008 and “agreed to update the site to accommodate sight-impaired online consumers, and to let the NFB regularly test those improvements once they are completed early next year,” according to Computerworld.

Despite this landmark settlement from a few years ago, accessibility testing still isn’t on the forefront of clients’ minds. The business case and value proposition is clear from the testing perspective and there are ways to easily work accessibility testing into your user experience (UX) or functional work.

In this uTest University webinar, accessibility testing expert Helen Burge discusses tips and tools for understanding accessibility testing, including the potential impact on the client’s reputation and the difference between accessibility, usability, and UX.

Some other topics covered in Helen’s “Tips and Tools for Understanding Accessibility Testing” webinar:

  • Web design often focuses on font choices for better readability, but color choices can affect how people consume the content.
  • Color ratios outlined by the Web Content Accessibility Guidelines (WCAG) 2.0 are an important checkpoint.
  • Testing tools range from screen readers and color analyzers, to Braille keyboards, HTML validators, and web developer toolbars.

Learn more about accessibility testing and other testing topics at uTest University, the learning hub for the testing community.

Continue Reading

Learn the Essentials of Accessibility Testing

uTest UniversityAt uTest, we specialize in in-the-wild testing. Our worldwide community of more than 100,000 testers provides real feedback from real users. But a truly great tester doesn’t limit themselves to issues that only they would encounter. They anticipate and seek out issues that other users would have as well. Making sure that an application doesn’t suffer from shorting access to users with physical or mental disabilities is at the heart of Accessibility testing. One of uTest’s top testers, Helen Burge, designed a uTest University course that takes us through a broad level overview of accessibility testing.

Accessibility testing, as Helen puts it, focuses on the user. Essentially, if your application cannot be used by someone who is visually impaired, hearing impaired, or has any number of other possible limitations, you have an accessibility issue. The accessibility testing course is a great tool for developers looking to avoid locking out certain users and for testers trying to suss out bugs related to accessibility. Helen runs through a number of scenarios where someone may be limited. She also provides a link the World Wide Web Consortium website where you can see internationally shared standards of accessibility.

The key takeaway from this course is that accessibility testing is just as, if not even more, important than its testing counterparts. While issues with functionality and usability may lead you to lose and alienate your customers, accessibility issues may lead to a lawsuit for discriminatory practices.

Continue Reading

Happy World Usability Day!

World Usability Day 2013For those of you that don’t already know, today is World Usability Day!  Yes, indeed, an entire day dedicated to celebrating the joy of simple, easy to use interfaces that make our day-to-day lives easier and more manageable. This year’s focus is on the healthcare industry, which is even more exciting!

What, you aren’t jumping with joy?  Well, let’s talk about usability and its role in healthcare and see if it puts you in a more celebratory mood.

Doctors, as we all know, don’t have a lot of time. They’re always busy, always on the move, and they see countless patients each and every day. The high demand and lowered supply of healthcare professionals coupled with the push to digital record keeping has given rise to the healthcare application as an industry all of its own. Which is great! Easy access to patient records, test results, chart data, and more right at a doctor’s fingertips means higher efficiency, lower costs, and happier doctors and patients alike, right? Right! But only if that access actually is easy.

Doctors, nurses, and other practitioners don’t have the time nor energy to be painstakingly sorting through oceans of buttons, links, and settings just to find the one thing they need right away. There’s no room for error, human or otherwise, when we’re talking about someone’s health – which in some cases is a life or death situation.

Healthcare is one of the few industries where poor usability affects not just the direct user of the app and its information, but also the receiver of that information. In essence, healthcare patients are secondary, or indirect, users of healthcare applications and their experience also matters.

You wouldn’t want to wait an extra hour for your doctor because they weren’t able to locate your e-chart, would you? And you certainly wouldn’t want to be given the incorrect diagnosis because your test results were buried in dust somewhere deep inside an application. Or to be given the wrong care, prescription, or prognosis just because the doctor’s tools were insufficiently laid out to meet his or her needs in any respect.

Healthcare applications need to work the first time, every time, in a way that is easy to use and understand at a glance. They need to be fast, accurate, stable, secure and intuitive. Not just for your doctor, but also for you.

So join with me and uTest in celebration of World Usability Day, not because usability is an amazing thing all around, but because good user experience really does affect our lives each and every day in a number of ways, and they should all be as great an experience as possible.

(Oh, and if you want our expert help making your user experiences the best they can be, take a peek at our brand new UX and Usability test offerings and our limited time usability testing discount.)

Continue Reading

5 Tips for Last Minute Usability Testing

Last minute usability testingInge De Bleecker is a uTest Usability Expert with more than 20 years experience with user interfaces and 13 years of experience with mobile usability. She’s a regular guest blogger on the uTest Software Testing Blog and in today’s post she’ll help you get your app ready for the holiday rush, even if you left usability testing until the tenth hour.


It is October, and the Fall season is here. Which means the end-of-year holidays are nearing. If you are part of a company that has an online presence, you may be working feverishly to get your site ready for the Holidays. Time is short and there is much to be done. And while usability testing has been on your mind, it might still be on the To Do list.

When it comes to usability, testing early in your development process and testing often along the way are key to your success. But that doesn’t mean that you can’t run a usability test in the tenth hour and still benefit. There is still time to identify some of the low-hanging fruit, and come out ahead.

Here is a list with some of the most common usability issues that can be quick and easy to fix:

Mobile web: According to a recent Pew Internet Research study, 64% of adult Americans go online using their mobile phone, and 21% use their mobile device primarily to go online, so it is important to have a user-friendly mobile experience. Go through your interface and ask yourself:

  • Can users easily tap on links? Links or buttons may be too close together or too small to allow for comfortable tapping.
  • Can users complete tasks on the mobile screen? Form elements can obscure the keyboard, making it impossible to complete account creation or check out on an e-tail site.

Call-to-Action: Your homepage or the first page users land on should explain to the user what they can use your site for, and how to get started. Check whether your main pages have a clear “Get Started” or “Do This Now” indicator. If not, users will leave your site promptly or wander around aimlessly.

Page layout: Items that belong together should be placed close to each other. Make sure that labels and instructions for input fields, such as a phone number format example, appear close to the input field. Dropdown arrows for accordion elements should be next to the section title rather than all the way off to the side.

Continue Reading →

Continue Reading

Did You HEAR? Usability is All in the Details

Apple Tri-ToneDid you know that a sound file entitled 158-marimba.aiff is played millions of times around the world each day? But what is it? Well it’s not Lady Gaga’s latest. Rather, it’s the ubiquitous iPhone alert known as “Tri-Tone.” But before it was the iPhone noise we either all love or hate, it was Apple’s way of telling you an app was installed. And before that it was the noise iTunes made to alert a user that a CD was done burning.

This short sound bite has a storied history but was originally developed by Kelly Jacklin for the pre-cursor to iTunes, an app called Sound Jam. Of course, the sound didn’t just fall out of mid-air. According to Jacklin, a lot of thought went into the environment the sound would be consumed in.

I was looking for something “simple” that would grab the user’s attention. I thought a simple sequence of notes, played with a clean-sounding instrument, would cut through the clutter of noise in a home or office. So I had two tasks: pick an instrument, and pick a sequence of notes. Simple, right? Yeah, says you; everyone’s an armchair musician. …

I was really into the sound of marimbas and kalimbas at the time, so I thought I’d try both of those. I also went through bank (after bank) of sounds built into the SW1000XG, auditioning instrument sounds, and found three other instrument sounds that I liked: a harp, a koto (Japanese zither), and a pizzicato string sound (that’s the sound a violinist makes when plucking the string, rather than bowing it).

For the notes, I wanted a 3-note sequence, or perhaps 4 notes. I was going for simple, and didn’t have much time to devote to being creative, so no fancy timing here, just sequenced notes. I wanted a happy feel, so notes from the major scale, focusing on I, III, IV, V, and VIII (the octave).

Now a normal person would have just started playing around on the keyboard. But I’m not normal, and decidedly not a keyboardist… So I went all “left brain” on this one (I’m normally used to avoiding that side of the brain when playing guitar or recording), and decided to write a program to generate the various permutations of the notes.

An afternoon of work later, 158-marimba.aiff was born alongside more than 25 alternatives. So why did Jacklin put forth the effort to create the tone? Simple, to create a memorable user experience. It’s details like the sounds chosen to alert users that take an app from good to great. Providing a tone to pierce through the noise clutter of an office, while still soothing the ear shows that the developer cares about the end user.

Of course, usability isn’t all about look and feel. Rather it’s about expectation and intuition. What do users expect out of an app, and how can a developer empower their users to understand how the app is being used? Paying attention to the details and putting yourself in the shoes of your user will enable you to create more fluid experiences.  So next time you’re questioning whether a workflow or interface feature needs polish or not, think back to Kelly Jacklin and the thought put into creating an easily identifiable and memorable sound to alert the user into action.

Continue Reading

The Importance of Accessibility Testing

Accessibility-NewsWhile many businesses are concerned with making sure their mobile, web and desktop applications don’t contain a mission-critical bug that’s a showstopper for launch, there’s oftentimes an important subset of users that gets glossed over in the process – those with disabilities or other impairments. Is the software accessible to all users before it hits the market?

For instance, for those that may be visually impaired, how are buttons in an app represented so screen readers can correctly read the information contained in the button?

It’s not just unfairly ignoring an entire slice of the population by not ensuring these quality standards –  it’s essentially ignoring the law. The Americans with Disabilities Act (ADA), for example, carries over to digital entities including mobile applications. The accessibility of mobile apps have to follow Web Content Accessibility Guidelines (WCAG) 2 standards including areas such as captioning and audio description.

Recently, there’s been a thought-provoking conversation in the uTest Forums that examines testers’ opinions on this important-but-oft-overlooked testing form. The uTester who started the conversation shared how her interest in accessibility testing was garnered from a personal experience when she was a student:

I had a friend with brittle bone disease and was blind. When she was trying to use Windows Media Player to play her music, she had to ask me to start the playlist as she had no way to navigate there herself….It shocked me to see how such a simple application was unusable to her, and the only “tool” she could use to start it was me!

One uTester echoed how affected she was by accessibility once actually immersed in a cycle:

I was also able to lead a testing effort for a mobile app that included accessibility, and months later, when I saw the users happy and loving the app, it was totally worth the effort. It was an eye-opener.

I’ll take a moment here and interject opinion. This was probably my favorite part of the discussion that responded to the naysayers of accessibility testing (because it hit the nail on the head):

I can see how some testers would consider it too much trouble and not want to do it, but then again, if you were to think that about your job, are you really a tester? A true tester would see this as an opportunity.

Accessibility testing is a very important variant of Usability testing and is something we take quite seriously here at uTest. It’s ultimately one of the most important and rewarding forms of testing that often gets lost in the shuffle when organizations roll out their applications for primetime.

Continue Reading

Guest Post: Windows 8 Design – Where Will It Take Us?

Windows 8Inge de Bleecker has been working as a User Interface Expert specializing in mobile for the past 12 years. Here’s her take on Windows 8 and why users may or may not ultimately embrace the usability change.

I recently was fortunate enough to attend a series of Windows 8 training sessions for user experience (UX) designers. Attending the sessions gave me valuable insight into Windows 8 design.

I am not here to give you a full list of the good, bad, and ugly of Windows 8. A quick Google search will reveal a number of links to articles that already cover that. What I am wondering about is what the Windows 8 user experience over time will evoke in its users.

The goal behind any good user experience has always been to provide the user with an interface that is easy and intuitive to use. With the advent of the iPhone, the interface also had to be fun to use. Usability and user delight are now top priorities.

During the Windows 8 training sessions, it quickly became clear that the Windows 8 user experience team has put a lot of thought into all aspects of user experience design. Based on that, they have created a vast number of rules; rules about what can be displayed on an app’s landing page, what needs to be stuck in menus, where menus reside on the page, the look and feel of page elements, and so on. There are so many rules that it is a bit overwhelming.

The good thing about rules is that it makes it much easier to design an app: just follow the rules, and the end result will be passable at the worst.

Consistent use of rules also leads to uniformity. For users over time, uniformity breeds familiarity. And familiarity promotes ease of use. To some extent.

While familiarity is one of the principles we strive for in design, discoverability is another one. I found that some of the Windows 8 design rules result in a lack of discoverability. To give an example, one of the rules states that actions should not be displayed on-screen, but instead provided in a (by default hidden) menu that can be opened through swiping. While users can become familiar with this and understand that they may need to look for something in a hidden menu, it impedes the discoverability. Yet usability testing time and again shows that users want actions at their fingertips, on-screen.

This example brings about some questions: Will familiarity in time overcome the discoverability issue and result in an experience that users find easy and intuitive? Will it make for an enjoyable experience?

My exposure to Windows 8 so far has definitely made me think. I am curious to see how users over time will rate the ease of use and enjoyability of Windows 8 applications. I’ll report my findings in this blog once I’ve gathered enough data. Stay tuned.

Continue Reading