Testing the Limits With Jakob Nielsen – Part I

What an honor it is to have Jakob Nielsen – the “King of Usability” – as our Testing the Limits guest this month. Jakob Nielsen, Ph.D. is principal of Nielsen Norman Group , a research and consulting firm that studies how people use technology. He is the author of many books, including Eyetracking Web Usability and Prioritizing Web Usability. He has invented several usability methods, including heuristic evaluation, and holds 79 United States patents, mainly on ways of making the Internet easier to use. For more, read his official biography.

In part I of our interview, we get his thoughts on the evolution of user experience; the superiority of native apps; tablet usability; the death of PDF files; iPhone vs. Android and other hot topics. Be sure to check back tomorrow for Part II of the interview. Enjoy!

uTest: Like everything, software usability is in a constant state of change. How have you managed to stay on top of a field that seems to get turned upside down every other month?

JN: The users keep me fresh. I don’t really have to know anything, because I can simply see what our test participants use and how they use it.

uTest: It seems to us that software usability is as much a study of human behavior than anything else. What other subjects would you advise people to study who want to learn more about user preferences? Psychology? Sociology? Others?

JN: The main thing I recommend is to study your actual users: invite a handful of representative customers to your location and run them through simple usability studies of your software. One day in the lab is worth a year in university lecture halls, in terms of actionable lessons learned. (And remember that your “usability lab” can be a regular office or conference room —as long as you shut the door.)

That said, it’s still well worth studying all branches of psychology (perceptual, cognitive, social, etc.). One of the most popular courses at the Usability Week conference is called “The Human Mind and Usability” and summarizes the most salient psych findings for designers who don’t have time to go back to school.

It’s also worth studying visual design, even if you’re never going to draw anything yourself. Knowing the concepts and language is helpful when communicating with graphic designers, both to let them know what you want and to understand their ideas.

uTest: In the world of mobile, there’s been a lot written on the subject of native apps vs. the mobile web. What’s your take on this debate? Do both methods have a role to play in the user landscape? And for companies just venturing the mobile realm, where would you tell them to focus their attention?

JN: Apps are superior for 3 reasons:

  • Empirically, users perform better with apps than with mobile sites in user testing.
  • Apps are much better at supporting disconnected use and poor connectivity, both of which will continue to be important use cases for years to come. When I’m in London and don’t feel like being robbed by “roaming” fees, any native mapping app will beat Google Maps at getting me to the British Museum.
  • Apps can be optimized for the specific hardware on each device. This will become more important in the future, as we get a broader range of devices.

Apps have the obvious downside of requiring more development resources, especially to be truly optimized for each device. If a company doesn’t have enough resources to do this right, it’s better to have a nice mobile site than a lame app.

A second downside of apps is that users have to install them. Our testing shows poor findability and usability in Apple’s Application Store, and many users won’t even bother downloading something at all for intermittent use. So ask yourself whether you’re really offering something within the hardcore mobile center of need: time-sensitive and/or location dependent, and whether your offer is truly compelling in this crowded space. Most companies are never going to make it big in mobile. In some cases all they need is to make their main website somewhat mobile-friendly. Many others should deliver a dedicated mobile site but not bother with apps.

uTest: Regarding tablets, we see a lot of companies taking their current iPhone app, increasing the graphic fidelity, and releasing it as an “original” iPad app. In your view, what the biggest mistake being made by companies developing apps specifically for tablet devices?

JN: The biggest problem in our recent tablet studies has been TMN: too much navigation. Also, too many inconsistently scrolling fields. Some tablet apps cram in so many weird features that users get overwhelmed and flail around without gaining mastery of the content.

I think that most designers of phone-based apps have recognized the need to limit the number of features and the number of wildly scrolling areas. The small screen imposes useful discipline that keeps out the worst excesses that still dilute usability on larger tablets.

While tablet UIs need to quiet down and become more consistent, that doesn’t mean that they should be phone designs with prettier graphics. The bigger screen allows for more features, and more focus on immersive use over longer periods of time than the quick hits that are most useful on phone-sized devices.

uTest: If you could eradicate one thing from the mainstream web to improve users’ experiences (eg: Flash, IE6, pop-ups, et al), what would it be and why?
JN: In 2000, it would definitely have been Flash. Today, it’s PDF.

uTest: Graduation season is coming up, which means a fresh batch of newly graduated graphic designers will be entering the workforce – if you could give them only one professional tip, what would it be?

JN: I know it’s boring, but the #1 tip for anybody involved with user interfaces is to do user testing: it’s invaluable to see how your design ideas work with real users. It’ll make your designs better, but it’ll also make you a better designer, because you’ll learn what’s real, as opposed to what sounded good in school.

uTest: With all of the focus now on mobile usability, have you seen any drop-off in the overall usability of traditional web and desktop apps? Is there a danger of these domains stagnating in the years ahead as mobile gets all the attention?

JN: No problem so far. While there’s always a danger, I think most companies realize that they cannot ignore their main website just because they get a mobile site or app. I usually emphasize that the usability guidelines for mobile are very different than the desktop guidelines, using this as an argument to design a separate mobile UI. But of course the argument cuts both ways: to be successful, you also need a separate desktop design. The desktop is where the vast majority of the money is made, except maybe for newspapers. Even when people use their mobile device for shopping, it’s often just for price comparisons and the actual purchase is made on a full-sized e-commerce site.

uTest: In terms of overall usability, what is your take on iPhone vs. Android? Is this a debate that will continue indefinitely? Or is one operating system in a better position (or putting its developers in a better position) to succeed with users?

JN: It’s a replay of the old tension between closed and open systems. Closed can be more tightly integrated and tends to have better usability because the entire system is architected as a whole. Open tends to be more unruly and confusing, but also offers a wider selection of choices. You could say Mac vs. Windows, or iOS vs. Android. Same basic issues. Windows won the former of these contests, and Android seems to be gaining ground in mobile.

The history lesson says that Windows eventually did emphasize usability and a stricter set of design guidelines starting around the time of Windows 95. But it’s taken them more than 15 years to overcome the legacy of bad usability in early Windows. You can’t just retrofit a strong user experience on top of a confused architecture. The peanut butter theory of design is false. (It claims that if you smear a thick enough layer on top, you can hide the taste of what’s below.)

In our current user testing, Android scores fairly well, but not as good as iPhone. For example, Android phones have several dedicated hard buttons (such as a menu button) that could theoretically help users do better than the iPhone with its single, overloaded button. Sadly, Android’s buttons are used inconsistently across devices and apps. As a result, users never know what each button will do, and thus never learn to use them or depend on them. Much stricter enforcement of design guidelines will be needed for Android to realize a usability advantage from its buttons.

That was one of the lessons from Macintosh software in the 1980s: developers who complied with the guidelines got better reviews and more customers, whereas inconsistent software got dinged by the Mac magazines. Because most early Mac software worked about the same, users could transfer their skills from one application to the next, and Mac owners bought about twice as many applications as PC owners.

Editor’s note: Read part II of the interview now.

Essential Guide to Mobile App Testing


  1. says

    Home owners who think they must be eligible for assistance can call 888-995-HOPE or visit
    the website to find out more on what help
    could be available. You can perform a fast phone around to various
    banks and building society’s and learn for yourself the minimum
    unexpired lease term each accepts. The more money that you simply are able to pay, the low your loan amount will be.

  2. says

    The success of windows and android’s “open” systems may have more to do with the economics (any vendor could create PC’s or Android devices, but only apple could manufacture apples) rather than user experience.

  3. says

    Great interview and a great resource. So good to see that Neilsen is keeping up with the latest trends in usability and UX. The Nielsen group recently updated an (updated) white paper on mobile usability that is well worth reading, lots of great principles.

    I agree with Dr. Nielsen that apps are, generally, more usable that mobile web, but the findability and ‘install-hassle issues cannot be overstated. Another one is app clutter, i.e. users do not want to have too many apps on their smartphones.

    I often recommend that clients should concentrate on their mobile web presence first and foremost, and only provide an app if they think it is something that their users really need.

  4. Nitin says

    I really thankfull to utest team for providing such a good interview. I picked up so many points which helped me in my project.thanks a lot utest team……………

  5. says

    The success of windows and android’s “open” systems may have more to do with the economics (any vendor could create PC’s or Android devices, but only apple could manufacture apples) rather than user experience.

  6. Dennis Dilley says

    I appreciated the article, especially the part about “usability guidelines for mobile are very different than the desktop guidelines” I have understood this myself and find it fitting that the writer has common sense as well as expertise. I also liked the part about the buttons in Android, which I had not known, and it makes sense.


Leave a Reply

Your email address will not be published. Required fields are marked *