Steve Jobs’ Advice for Software Testers

Actually, in this 2005 commencement speech for Stanford University, Steve Jobs offers timeless advice for people in all professions. But seeing how this is a software testing blog – and seeing how Jobs has recently stepped down as Apple’s CEO – it seemed fitting to post these words of wisdom with our testing audience in mind.

My favorite quote from this speech: “The only way to do great work, is to love the work that you do.”

Happy testing!

iOS Developers Love iOS, Maybe Not OS X

After the conclusion of this year’s Apple WWDC conference, Gene Munster of Piper Jaffray released the results of an informal survey he performed among conference attendees who were also iOS developers. In it, he asked them what their plans were for developing on different platforms, including Apple’s own OS X. The results were surprising.

iOS developers love iOS (of course), and as recently as 2008, 50% of them were also OS X developers. But today, that percentage has dropped to 7%, and most iOS developers are now actively developing for other platforms instead (including the iPad). This makes a lot of sense – the skillset for developing a mobile application has become more and more specialized, and the developers who can do that well may not have the skills or interest in developing for a desktop platform.

But the data holds other clues as well. For example, almost half of iOS developers also develop for Android. And even though all the developers think iOS is the best platform for monetization (they were attending WWDC after all), 40% of them thought Android was the platform with the greatest potential for future growth. By the way, that question included iOS as an option as well, meaning that 40% of iOS developers attending WWDC actually thought Android was going to grow faster than iOS.

What other platforms did these developers think would have any chance of growing in the coming years? The only other one that made the list was Windows Phone 7 with 9% of respondents. That’s small, but interesting. Microsoft could have something good on their hands.

More details from Fortune and Macrumors.

Bug Roundup – News From the Week

We love studying bugs when they come up, and this past week we’ve seen a few big ones go by. When bugs happen, there’s always a lot we can learn from them. Here’s a quick roundup of four different bugs that were recently in the news:

Apple iPhone Tracking – First up, we learned last week that iPhones store their location in a file that never gets deleted, and then backup that file to iTunes each time the phone syncs. That means that anyone with access to a laptop belonging to an iPhone owner could see where they had been as long as they had owned their phone. (For the record, my iPhone says I spend a lot of time in Southborough, MA at the uTest headquarters.)

After a few days of silence on the issue, Apple announced that this was the result of a bug in iOS – three bugs actually. 1 – the iPhone keeps the location data for too long and should instead periodically purge it. 2 – this data is backed up to iTunes and should not be. 3 – the data is not deleted if a user disables location services. Apple has plans to fix all three bugs and to also begin encrypting the location file on the iPhone.

Why were they tracking this data at all? Apple uses this information (anonymously) to improve their location services and make it easier for iPhones to determine their location without having to resort to GPS (which is slow). But they only need a small amount of data at a time rather than the entire location history the iPhone was storing.

Do you have an iPhone? Are you curious to see where you’ve been? Here’s a clever app that will plot your location history on a map. If you’re into fancy statistical analysis, you can also use this add-on to plot your location using R.

Continue Reading

Testing the Limits With Jakob Nielsen – Part I

What an honor it is to have Jakob Nielsen – the “King of Usability” – as our Testing the Limits guest this month. Jakob Nielsen, Ph.D. is principal of Nielsen Norman Group , a research and consulting firm that studies how people use technology. He is the author of many books, including Eyetracking Web Usability and Prioritizing Web Usability. He has invented several usability methods, including heuristic evaluation, and holds 79 United States patents, mainly on ways of making the Internet easier to use. For more, read his official biography.

In part I of our interview, we get his thoughts on the evolution of user experience; the superiority of native apps; tablet usability; the death of PDF files; iPhone vs. Android and other hot topics. Be sure to check back tomorrow for Part II of the interview. Enjoy!

uTest: Like everything, software usability is in a constant state of change. How have you managed to stay on top of a field that seems to get turned upside down every other month?

JN: The users keep me fresh. I don’t really have to know anything, because I can simply see what our test participants use and how they use it.

uTest: It seems to us that software usability is as much a study of human behavior than anything else. What other subjects would you advise people to study who want to learn more about user preferences? Psychology? Sociology? Others?

JN: The main thing I recommend is to study your actual users: invite a handful of representative customers to your location and run them through simple usability studies of your software. One day in the lab is worth a year in university lecture halls, in terms of actionable lessons learned. (And remember that your “usability lab” can be a regular office or conference room —as long as you shut the door.)

That said, it’s still well worth studying all branches of psychology (perceptual, cognitive, social, etc.). One of the most popular courses at the Usability Week conference is called “The Human Mind and Usability” and summarizes the most salient psych findings for designers who don’t have time to go back to school.

It’s also worth studying visual design, even if you’re never going to draw anything yourself. Knowing the concepts and language is helpful when communicating with graphic designers, both to let them know what you want and to understand their ideas.

uTest: In the world of mobile, there’s been a lot written on the subject of native apps vs. the mobile web. What’s your take on this debate? Do both methods have a role to play in the user landscape? And for companies just venturing the mobile realm, where would you tell them to focus their attention?

JN: Apps are superior for 3 reasons:

  • Empirically, users perform better with apps than with mobile sites in user testing.
  • Apps are much better at supporting disconnected use and poor connectivity, both of which will continue to be important use cases for years to come. When I’m in London and don’t feel like being robbed by “roaming” fees, any native mapping app will beat Google Maps at getting me to the British Museum.
  • Apps can be optimized for the specific hardware on each device. This will become more important in the future, as we get a broader range of devices.

Apps have the obvious downside of requiring more development resources, especially to be truly optimized for each device. If a company doesn’t have enough resources to do this right, it’s better to have a nice mobile site than a lame app.

A second downside of apps is that users have to install them. Our testing shows poor findability and usability in Apple’s Application Store, and many users won’t even bother downloading something at all for intermittent use. So ask yourself whether you’re really offering something within the hardcore mobile center of need: time-sensitive and/or location dependent, and whether your offer is truly compelling in this crowded space. Most companies are never going to make it big in mobile. In some cases all they need is to make their main website somewhat mobile-friendly. Many others should deliver a dedicated mobile site but not bother with apps.

uTest: Regarding tablets, we see a lot of companies taking their current iPhone app, increasing the graphic fidelity, and releasing it as an “original” iPad app. In your view, what the biggest mistake being made by companies developing apps specifically for tablet devices?

Continue Reading

Apple’s iPad 2 Release Date & Information

There are only two kinds of people who aren’t following the iPad 2 saga with every waking moment – luddites and first gen iPad owners.  I fall into the second category.

Before we get into the iPad 2 details, a quick rant if I may.  Apple has long been criticized (yes, that’s an Oatmeal link) for having little loyalty to early adopters. Price drops, product launches, and planned obsolescence are often called as being pre-set in a manner in which to capitalize profits at the benefit of the company.  Last I checked Apple is a for-profit company.  As an advertiser I envy the customer loyalty that Apple has built that is able to create such a premium for early adopters.

The biggest problem with the common conspiracy theory that Apple is intentionally delaying features in products is that Google is making a huge dent in Apple’s smart phone market share. Apple wouldn’t allow that if they had the pipeline – or ability – to crank out products faster.  I’d love to continue this debate in the comments section below – but for now, I digress.

There’s a lot of news articles claiming today is the iPad 2 release day.  To be perfectly clear, today is the announcement from Apple but iPad 2’s won’t be available for purchase until next week.

So what’s different about the iPad 2?

Continue Reading

Google’s Whirlwind Start To 2011

Google’s off to a pretty intense start in 2011 – from a change in CEO to launching new products that compete directly with some of the biggest tech companies including Microsoft, Amazon and of course, Apple.

It’s no secret that web and mobile apps represent a lot of money to businesses and app markets are in a race to keep up. Google is using this as an opportunity to greatly expand their presence — and the early returns are impressive. In fact, the Android app market is growing 3x faster than Apple’s iOS market (although, as its marketshare grows, it become a more attractive target to black hat malware apps).

Google isn’t stopping there, though. They’ve recently launched their Shopper app on iOS – an alternative to Amazon’s really nice native apps – and the “One Pass” a publisher subscription alternative to Apple.

Continue Reading

Symbian And BlackBerry Hangin’ Tough as Android Takes Over

According to Mashable and research firm Canalys, Android overtook Symbian to become the world’s most popular smartphone platform in Q4 2010.

Out of the 101.2 million Q4 worldwide shipments of smartphones, Android claimed 33.3 million compared to Symbian’s 31 million. Apple’s iOS took the bronze with 16.2 million smartphone shipments, followed by RIM with 14.6 million, and Microsoft rounds out the list with 3.1 million devices shipped.

There is no denying that, just as Apple’s iOS revolutionized the smartphone category, Android has come on the scene and made major waves as well. Take a look at the graphs and the article here, both courtesy of Nielsen, to see this impact in vivid color.

How will the recently launched Verizon iPhone impact smartphone market shares? It’s not clear right now, but if pre-order reports are any indication, it could be quite significant. And what about the impact of Windows Phone 7 OS? Still too early to tell.

What is clear, is that some of the one-time industry leaders are being challenged and even surpassed. Unless they respond with new and better versions, they’ll be left on the sidelines to watch as the new kids on the block (no, not the real NKOTB) take over. All we can say for those who are looking up at the leaders is to keep “hangin’ tough.”

uTest iPhone and iPad App – Test On the Go

When we relaunched our testing platform last year, we chose to build it in Flex. It allowed us to build a nice web UI, but it also meant that iPhone and iPad owners couldn’t connect to our platform directly. An Apple iOS user had to find a regular computer to report bugs when testing an iOS app.

Well we’ve heard their pleas, and we’re happy to announce something that should help: the uTest iOS app. With a native interface for both the iPhone and the iPad, it’s now possible for testers and customers to test on the couch and on the go.

If that’s enough to make you want to download the app right now, then don’t let me stand in your way. Just click (or tap) on that image to the left and go get it. It’s free, you know.

If you’re still wondering what makes our app special, let me tell you about some of the awesome new things that both testers and customers can do.

For Testers

Customers and testers can start testing with the uTest iPhone app.

iPad users have a native interface that makes full use of the iPad layout.

It goes without saying that the uTest iOS app lets you participate in test cycles and that our app makes it easy to submit bugs right from your iPhone or iPad. But what’s really cool is that if you’re testing another iOS app, you can submit screenshots and videos of your bugs directly from the uTest app. You can even use your camera to take pictures for upload – handy if you need a screenshot of a bug on another mobile device.

In addition to all that, you can do all the other things you would expect while testing, like view the bugs submitted by other testers, reply to tester messenger conversations, and even check out your uTest earnings.

Of course, all this assumes you’re already a uTester. Because if you’re not, you can actually signup for a tester account right there in the app. It will even help you setup your iPhone or iPad as your first testing device on the uTest platform.

For Customers
If you’re a uTest customer, you’re going to love the uTest app. Why approve and reject bugs from a boring old computer when you can do it from the beach? And if you don’t have a beach nearby, how about the comfort of your couch? In fact, you can now review your test cycles from anywhere you like (assuming there’s a phone or wireless signal, of course).

With the uTest app, you’ll also be able to review attachments and even ask testers questions with tester messenger. Everything you need to keep an eye on a test cycle is available at your fingertips.

Wrapup
Of course, our community rigorously tested our iOS app and they discovered over 60 bugs before launch. Their diligence made this app super solid, and that helped us to get approved by Apple for the App Store in record time without having to resubmit.

Now that we’ve launched our first iOS app, we’re hardly finished. We want your feedback and ideas about how we can make it even better. uTest community members can join our tester forums and check out our Platform Feedback section. Customers can contact their project manager directly or drop us a line.

Are You Buying a Verizon iPhone?

Here in the United States, iPhone users have long complained about the quality of service from AT&T. Being the nation’s largest GSM carrier, AT&T was the logical first choice for the iPhone when it launched. Apple could reach a large population of Americans and then expand globally, all using the same device.

But AT&T has a mixed track record of keeping up with the demands of the iPhone. In some parts of the country, their service is great. In other parts, it’s pretty terrible. Many AT&T customers have long wanted to switch to America’s other big phone network: Verizon. The problem with Verizon is that it uses a completely different cellular phone standard called CDMA. Using the iPhone on Verizon required a different hardware design, and that was only after Apple got out of their exclusive deal with AT&T for selling the iPhone.

Today both Verizon and Apple finally delivered: the long awaited CDMA iPhone. Starting February 3, Verizon customers can start using the iPhone on America’s other big network. Are you planning to get a Verizon iPhone?

[poll id=”4″]

Verizon owners will mostly have the same phone experience, but with a few small changes. Continue Reading

The iPhone Alarm Clock Glitch: Lessons In Bug Reporting

Whenever a software bug is submitted, one of the first things a testing manager or developer should ask is this: Is this defect reproducible? If the tester has written a clear and concise bug report, it will contain a short description of the expected result, the actual result, and the specific steps required to reproduce the defect. It will also contain diagnostic information such as bug type, bug severity and bug frequency. There’s a little more to it than that, of course, but you get the idea.

Reports that don’t contain this type of information are likely to be ignored or dismissed by developers – and rightly so – since they aren’t in the business of “taking your word for it.” That said, sometimes even a perfectly-worded, crystal-clear bug report can slip through the cracks, make front page headlines across the globe, and tarnish an otherwise stellar reputation for quality.

I would have to put Apple’s latest iPhone bug – soon to be known as, ugh, Alarm-Gate – in that category. I’ll spare you the full details (for that you should read this post on The LA Times blog) except to say that the iPhone alarm clock has been malfunctioning across the globe for the last three days. It’s caused missed flights,  no-shows for the first day of work and thousands of angry tweets.

So why, without a shred of evidence, would I suggest that this bug was discovered by someone at Apple and dismissed or ignored? Part of it has to do with the fact that the app must have been tested at some point, which would have had to included a use case such as a year change. Fair enough? Part of it has to do with the lessons from Antenna-Gate, including the engineer who had reported the issue well in advance of the product’s launch. But most of it has to do with the nature of tester-client interactions.

uTester Bill Ricardi explains it well:

Continue Reading