Our guest in this installment of Testing the Limits is Craig Tomlin, an award-winning digital marketing and User Experience (UX) consultant with over 20 years’ experience in B2B and B2C demand generation and eCommerce. He leads marketing and UX strategies and tactics for firms like: Blue Cross Blue Shield, BMC Software, Disney, IBM, Kodak, Prudential, WellPoint and more. You can follow Craig on Twitter @ctomlin.
In the first part of this two-part interview, Craig talks the differences between Usability and functional testing, and why Usability often doesn’t get the spotlight it deserves.
uTest: What initially drew you into exploring usability?
CT: I was drawn to usability in the mid-1990s, when preparing to redesign 22 websites for WellPoint Health Networks, a major health insurance corporation in the U.S. My earlier attempts at redesigning sites sometimes failed miserably. I didn’t understand why. After all, we had brilliant I.T. teams, the best website design vendors, internal stakeholders that knew their products and subject matter backwards and forwards. So why the failures? I realized the missing ingredient was the users.
Without user-centered design, we were just guessing about important usability issues when designing new experiences. I took it upon myself to become certified in usability and apply user-centered design principles to the 22 website redesign project. We conducted extensive usability testing, and created mental maps, information architectures and taxonomies based on user-defined requirements. Because of the inclusion of users as part of the process, we were very successful with our new designs. I’ve been a big fan of usability and user-centered design ever since.
uTest: When folks think of usability, it can often get muddied or confused with the functionality of a site. What goes into a Usability Audit?
CT: The International Organization for Standardization (ISO) defines usability as “The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.” Therefore, usability includes both functionality (the effectiveness and efficiency portion) and satisfaction, of which both can be measured using quantifiable data. Usability testing if done properly includes evaluations of the critical tasks associated with a website or app in conjunction with the overall satisfaction of that experience.
As to a usability audit, it is an evaluation of effectiveness, efficiency and satisfaction using a well-defined set of criteria that can be reproduced over any number of websites, apps or other objects. I think that strictly speaking, functional testing only evaluates that effectiveness portion of the definition, in that a function either works as specified, or does not work as specified. Any data on efficiency and satisfaction would typically not be part of a functional test.
uTest: You’re one of 4,900 certified usability analysts in the world. How does one get certified, and what differentiates a certified usability analyst from an ordinary tester who’s providing their off-the-cuff feedback about a site or app?
CT: Becoming a certified usability analyst was very beneficial to me, my clients and the people who use the websites and applications I create. It’s interesting that accountants, doctors, lawyers and even hair stylists all have to become qualified through passing exams before they start dealing with real clients. Isn’t it sad that anyone can call themselves a ‘usability expert’ even though they have absolutely no training at all in usability? It’s a real case of ‘buyer beware’ for anyone seeking a knowledgeable usability expert to help them improve their website or app user experience. I’ve seen plenty of examples of untrained usability advisors providing bad advice to clients that caused them harm.
Being certified means that person has taken the time to learn the appropriate skills to properly test usability principles and a mark of someone who not only talks the talk, but also walks the walk. There are multiple ways to become educated in usability, including taking usability and human computer interaction courses at universities, or doing what I did and becoming certified through the Human Factors International’s CUA program. The 4,900 or so CUAs in the HFI program all completed an extensive set of educational programs and then passed the test (it took me two and a half hours to complete). I highly recommend either going through a university program or taking the CUA course to become a proficient usability analyst.
uTest: Do you find that usability often falls beneath making sure a site ‘just works’ for many organizations?
CT: Yes. I wish I could say that most organizations today value usability and user-centered design. However, every day I come across a website or app that clearly has significant usability issues, a sign that a firm that did not value or use usability in their development process. More than likely, the creators of those websites wanted to provide the best user experience possible, but based on budget, resource or time constraints did not use usability as part of the software development process.
Many times, firms resort to usability only AFTER they’ve launched a new app or site, and discovered that their conversion is much lower than they expected. Reactive usability never works as well as proactive usability. Firms that do it right are including user-centered design throughout the software development lifecycle.
uTest: What skills must a usability expert have that a functional tester wouldn’t be concerned with?
CT: Skill sets can vary widely between a functional tester and a usability expert. Functional testing has extremely specific criteria to evaluate, and typically the result of testing is a ‘yes’ or ‘no’ answer. Did the function work as specified?
Usability testing seeks to understand where there are hurdles or failure points in a critical task flow, beyond just a unique function. Most often, that critical task includes many different functions, and sometimes no function at all. Usability results provide insights into where task flow failure is occurring across a system, and provide recommendations for optimizations of elements of the system as well as the holistic experience. So a usability expert must evaluate the holistic system, and elements of that system as opposed to evaluating if a specific function does or does not work to specification.
For example, often usability issues are caused by poor labeling, illogical navigation or placing content or objects in places that do not align with a typical user’s mental map. Those issues have nothing to do with the functionality of tools or applications on a site, and everything to do with the designer’s lack of understanding of the user’s viewpoint of what things should be called, how to navigate to find those things and where those things should be placed. A website or application might function flawlessly, yet still be completely unusable by the intended audience.
Stay tuned for Part II of the interview with Craig next week.