The Trouble With Testing Mobile

We hear it over and over from countless customers, QA managers and testers:  getting mobile apps tested is tough.

Compared to mobile, the worlds of web and desktop apps are mature and stable:  standards have emerged, the OS and browser universe is more mature, testing tools and best practices have been established (even though they’re always evolving).  Mobile, on the other hand, is still early in its overall lifecycle, so development and testing aren’t yet fully baked.

With that in mind, I came across an article from CNET yesterday titled, Testing the iPhone 3Gs (or why the phone gods hate product testers.  This piece underscores the difficulties that confront companies (even the well-equipped experts at the CNET labs) when trying to test mobile devices and apps.

Eric Franklin, a senior tech editor at CNET, was trying to test the talk time on the new 3Gs… sounds simple enough,  right?  Well, in Eric’s own words, he encountered a number of “roadblocks” and was “slapped back to reality” on more than one occasion.

As mobile matures, more development tools and standards will emerge.  And in time, testing tools and bulletproof practices will evolve.  Until then, building and testing mobile apps for the iPhone, Blackberry, Android and Palm Pre will continue to feel a bit like the wild west.  Take a look at the below pic from CNET’s labs… eerily similar to the ad hoc way many companies developed/testedwebsites and apps in the late 90′s.

cnet-mobile-test2

The iPhone 3GS during a talk time test. The "Watchmen" DVD is merely there to simulate a user holding the phone to their ear, thereby shutting off the screen. Music from the Zune is being played through both phone mics. (Credit: Eric Franklin/CNET)

What say you?  How do you or your company managed the ever-changing world of mobile without launching something shoddy and disappointing (read as: losing) your customers?

Essential Guide to Mobile App Testing

Leave a Reply

Your email address will not be published. Required fields are marked *