Testing the Limits With Jakob Nielsen – Part I

What an honor it is to have Jakob Nielsen – the “King of Usability” – as our Testing the Limits guest this month. Jakob Nielsen, Ph.D. is principal of Nielsen Norman Group , a research and consulting firm that studies how people use technology. He is the author of many books, including Eyetracking Web Usability and Prioritizing Web Usability. He has invented several usability methods, including heuristic evaluation, and holds 79 United States patents, mainly on ways of making the Internet easier to use. For more, read his official biography.

In part I of our interview, we get his thoughts on the evolution of user experience; the superiority of native apps; tablet usability; the death of PDF files; iPhone vs. Android and other hot topics. Be sure to check back tomorrow for Part II of the interview. Enjoy!

uTest: Like everything, software usability is in a constant state of change. How have you managed to stay on top of a field that seems to get turned upside down every other month?

JN: The users keep me fresh. I don’t really have to know anything, because I can simply see what our test participants use and how they use it.

uTest: It seems to us that software usability is as much a study of human behavior than anything else. What other subjects would you advise people to study who want to learn more about user preferences? Psychology? Sociology? Others?

JN: The main thing I recommend is to study your actual users: invite a handful of representative customers to your location and run them through simple usability studies of your software. One day in the lab is worth a year in university lecture halls, in terms of actionable lessons learned. (And remember that your “usability lab” can be a regular office or conference room —as long as you shut the door.)

That said, it’s still well worth studying all branches of psychology (perceptual, cognitive, social, etc.). One of the most popular courses at the Usability Week conference is called “The Human Mind and Usability” and summarizes the most salient psych findings for designers who don’t have time to go back to school.

It’s also worth studying visual design, even if you’re never going to draw anything yourself. Knowing the concepts and language is helpful when communicating with graphic designers, both to let them know what you want and to understand their ideas.

uTest: In the world of mobile, there’s been a lot written on the subject of native apps vs. the mobile web. What’s your take on this debate? Do both methods have a role to play in the user landscape? And for companies just venturing the mobile realm, where would you tell them to focus their attention?

JN: Apps are superior for 3 reasons:

  • Empirically, users perform better with apps than with mobile sites in user testing.
  • Apps are much better at supporting disconnected use and poor connectivity, both of which will continue to be important use cases for years to come. When I’m in London and don’t feel like being robbed by “roaming” fees, any native mapping app will beat Google Maps at getting me to the British Museum.
  • Apps can be optimized for the specific hardware on each device. This will become more important in the future, as we get a broader range of devices.

Apps have the obvious downside of requiring more development resources, especially to be truly optimized for each device. If a company doesn’t have enough resources to do this right, it’s better to have a nice mobile site than a lame app.

A second downside of apps is that users have to install them. Our testing shows poor findability and usability in Apple’s Application Store, and many users won’t even bother downloading something at all for intermittent use. So ask yourself whether you’re really offering something within the hardcore mobile center of need: time-sensitive and/or location dependent, and whether your offer is truly compelling in this crowded space. Most companies are never going to make it big in mobile. In some cases all they need is to make their main website somewhat mobile-friendly. Many others should deliver a dedicated mobile site but not bother with apps.

uTest: Regarding tablets, we see a lot of companies taking their current iPhone app, increasing the graphic fidelity, and releasing it as an “original” iPad app. In your view, what the biggest mistake being made by companies developing apps specifically for tablet devices?

Continue Reading

Apple’s iPad 2 Release Date & Information

There are only two kinds of people who aren’t following the iPad 2 saga with every waking moment – luddites and first gen iPad owners.  I fall into the second category.

Before we get into the iPad 2 details, a quick rant if I may.  Apple has long been criticized (yes, that’s an Oatmeal link) for having little loyalty to early adopters. Price drops, product launches, and planned obsolescence are often called as being pre-set in a manner in which to capitalize profits at the benefit of the company.  Last I checked Apple is a for-profit company.  As an advertiser I envy the customer loyalty that Apple has built that is able to create such a premium for early adopters.

The biggest problem with the common conspiracy theory that Apple is intentionally delaying features in products is that Google is making a huge dent in Apple’s smart phone market share. Apple wouldn’t allow that if they had the pipeline – or ability – to crank out products faster.  I’d love to continue this debate in the comments section below – but for now, I digress.

There’s a lot of news articles claiming today is the iPad 2 release day.  To be perfectly clear, today is the announcement from Apple but iPad 2’s won’t be available for purchase until next week.

So what’s different about the iPad 2?

Continue Reading

Google’s Whirlwind Start To 2011

Google’s off to a pretty intense start in 2011 – from a change in CEO to launching new products that compete directly with some of the biggest tech companies including Microsoft, Amazon and of course, Apple.

It’s no secret that web and mobile apps represent a lot of money to businesses and app markets are in a race to keep up. Google is using this as an opportunity to greatly expand their presence — and the early returns are impressive. In fact, the Android app market is growing 3x faster than Apple’s iOS market (although, as its marketshare grows, it become a more attractive target to black hat malware apps).

Google isn’t stopping there, though. They’ve recently launched their Shopper app on iOS – an alternative to Amazon’s really nice native apps – and the “One Pass” a publisher subscription alternative to Apple.

Continue Reading

Symbian And BlackBerry Hangin’ Tough as Android Takes Over

According to Mashable and research firm Canalys, Android overtook Symbian to become the world’s most popular smartphone platform in Q4 2010.

Out of the 101.2 million Q4 worldwide shipments of smartphones, Android claimed 33.3 million compared to Symbian’s 31 million. Apple’s iOS took the bronze with 16.2 million smartphone shipments, followed by RIM with 14.6 million, and Microsoft rounds out the list with 3.1 million devices shipped.

There is no denying that, just as Apple’s iOS revolutionized the smartphone category, Android has come on the scene and made major waves as well. Take a look at the graphs and the article here, both courtesy of Nielsen, to see this impact in vivid color.

How will the recently launched Verizon iPhone impact smartphone market shares? It’s not clear right now, but if pre-order reports are any indication, it could be quite significant. And what about the impact of Windows Phone 7 OS? Still too early to tell.

What is clear, is that some of the one-time industry leaders are being challenged and even surpassed. Unless they respond with new and better versions, they’ll be left on the sidelines to watch as the new kids on the block (no, not the real NKOTB) take over. All we can say for those who are looking up at the leaders is to keep “hangin’ tough.”

uTest iPhone and iPad App – Test On the Go

When we relaunched our testing platform last year, we chose to build it in Flex. It allowed us to build a nice web UI, but it also meant that iPhone and iPad owners couldn’t connect to our platform directly. An Apple iOS user had to find a regular computer to report bugs when testing an iOS app.

Well we’ve heard their pleas, and we’re happy to announce something that should help: the uTest iOS app. With a native interface for both the iPhone and the iPad, it’s now possible for testers and customers to test on the couch and on the go.

If that’s enough to make you want to download the app right now, then don’t let me stand in your way. Just click (or tap) on that image to the left and go get it. It’s free, you know.

If you’re still wondering what makes our app special, let me tell you about some of the awesome new things that both testers and customers can do.

For Testers

Customers and testers can start testing with the uTest iPhone app.

iPad users have a native interface that makes full use of the iPad layout.

It goes without saying that the uTest iOS app lets you participate in test cycles and that our app makes it easy to submit bugs right from your iPhone or iPad. But what’s really cool is that if you’re testing another iOS app, you can submit screenshots and videos of your bugs directly from the uTest app. You can even use your camera to take pictures for upload – handy if you need a screenshot of a bug on another mobile device.

In addition to all that, you can do all the other things you would expect while testing, like view the bugs submitted by other testers, reply to tester messenger conversations, and even check out your uTest earnings.

Of course, all this assumes you’re already a uTester. Because if you’re not, you can actually signup for a tester account right there in the app. It will even help you setup your iPhone or iPad as your first testing device on the uTest platform.

For Customers
If you’re a uTest customer, you’re going to love the uTest app. Why approve and reject bugs from a boring old computer when you can do it from the beach? And if you don’t have a beach nearby, how about the comfort of your couch? In fact, you can now review your test cycles from anywhere you like (assuming there’s a phone or wireless signal, of course).

With the uTest app, you’ll also be able to review attachments and even ask testers questions with tester messenger. Everything you need to keep an eye on a test cycle is available at your fingertips.

Wrapup
Of course, our community rigorously tested our iOS app and they discovered over 60 bugs before launch. Their diligence made this app super solid, and that helped us to get approved by Apple for the App Store in record time without having to resubmit.

Now that we’ve launched our first iOS app, we’re hardly finished. We want your feedback and ideas about how we can make it even better. uTest community members can join our tester forums and check out our Platform Feedback section. Customers can contact their project manager directly or drop us a line.

Are You Buying a Verizon iPhone?

Here in the United States, iPhone users have long complained about the quality of service from AT&T. Being the nation’s largest GSM carrier, AT&T was the logical first choice for the iPhone when it launched. Apple could reach a large population of Americans and then expand globally, all using the same device.

But AT&T has a mixed track record of keeping up with the demands of the iPhone. In some parts of the country, their service is great. In other parts, it’s pretty terrible. Many AT&T customers have long wanted to switch to America’s other big phone network: Verizon. The problem with Verizon is that it uses a completely different cellular phone standard called CDMA. Using the iPhone on Verizon required a different hardware design, and that was only after Apple got out of their exclusive deal with AT&T for selling the iPhone.

Today both Verizon and Apple finally delivered: the long awaited CDMA iPhone. Starting February 3, Verizon customers can start using the iPhone on America’s other big network. Are you planning to get a Verizon iPhone?

[poll id="4"]

Verizon owners will mostly have the same phone experience, but with a few small changes. Continue Reading

The iPhone Alarm Clock Glitch: Lessons In Bug Reporting

Whenever a software bug is submitted, one of the first things a testing manager or developer should ask is this: Is this defect reproducible? If the tester has written a clear and concise bug report, it will contain a short description of the expected result, the actual result, and the specific steps required to reproduce the defect. It will also contain diagnostic information such as bug type, bug severity and bug frequency. There’s a little more to it than that, of course, but you get the idea.

Reports that don’t contain this type of information are likely to be ignored or dismissed by developers – and rightly so – since they aren’t in the business of “taking your word for it.” That said, sometimes even a perfectly-worded, crystal-clear bug report can slip through the cracks, make front page headlines across the globe, and tarnish an otherwise stellar reputation for quality.

I would have to put Apple’s latest iPhone bug – soon to be known as, ugh, Alarm-Gate – in that category. I’ll spare you the full details (for that you should read this post on The LA Times blog) except to say that the iPhone alarm clock has been malfunctioning across the globe for the last three days. It’s caused missed flights,  no-shows for the first day of work and thousands of angry tweets.

So why, without a shred of evidence, would I suggest that this bug was discovered by someone at Apple and dismissed or ignored? Part of it has to do with the fact that the app must have been tested at some point, which would have had to included a use case such as a year change. Fair enough? Part of it has to do with the lessons from Antenna-Gate, including the engineer who had reported the issue well in advance of the product’s launch. But most of it has to do with the nature of tester-client interactions.

uTester Bill Ricardi explains it well:

Continue Reading

We Really, Really Like Feature Bloat

Here’s a test: let’s say you’re comparing microwaves. You want a good one, but you don’t know anything about the brands. So you go visit the appliance store and see two models – one that’s basic looking and one with lots of buttons and displays. Which one do you like better?

Chances are, you’ll like the microwave with all the extra buttons. People like features, and despite calls for “simplicity in design” we actually choose features over simplicity all the time. For example, I recently had my car in for repairs, and during that time I drove a loaner car that was one year newer. The newer loaner had some nice features that I now really miss on my older, simpler car. I love features.

Yet when we talk about usability, we always talk about “simplicity.” The mantra of good design is to say or do exactly enough, but no more. Don’t over complicate things. We do this because we believe that simplicity is the way to drive adoption, in large part because Apple preaches it with their incredibly successful products. But the usability experts may be wrong, and that’s a problem.

Continue Reading

New Apple Stuff – iOS 4.1, iTunes Ping, iPods, and More!

When Steve Jobs stands on stage in front of an audience, you can guarantee you’ll get a show. Occasionally it’s a show like the iPhone 4 antenna presentation from this July where Apple was at its prickliest. But yesterday it was the more exciting kind of show – new products!

So what did Santa Steve deliver to all of us good little boys and girls? Let’s check out the highlights:

iOS 4.1 and iOS 4.2
Probably the biggest deal to developers and testers is the upcoming release of iOS 4.1 (rumored to be launching next week on September 8). It’s no secret that iOS 4 has been buggy with problems like flaky Bluetooth support, slow performance, and random crashes. So it was welcome news to hear that iOS 4.1 includes a substantial number of bug fixes. Even the infamous proximity sensor bug has been fixed, which is fantastic because just days ago it sounded like that fix would be delayed.

Apple is also introducing a new social gaming network called Game Center. This new platform will provide developers with an easy to integrate social network for their games that will let players track achievements and play multiplayer games online with friends.

Continue Reading

Testing Stories From Developing MacPaint

Creating new platforms like Android and iPhone is incredibly difficult, but it’s rare to hear stories about the challenges of building them unless you’re an insider.  There are probably dozens of good tales about developing these platforms that will take years to trickle out from behind closed doors.

So to hear stories like these, we must look back in time at the great development projects of the past.  Today the Computer History Museum announced that Apple has donated the source code for the original MacPaint application so that it can be downloaded by anyone.  MacPaint was a drawing application included with the first Macintosh that by today’s standards seems very simple, but in 1984 was completely revolutionary.  Many of the graphic design tools we take for granted, like the paint bucket and lasso select, were invented in MacPaint.

For developers and testers alike, there’s a lot to learn from the development of MacPaint.  Here are a few good stories:

Continue Reading