uTest At STPCon 2011: A Recap

A great time had by all at STPcon 2011 down in Nashville, Tennessee. For those not able to attend (and those who were not following STP’s live stream) I wanted to post a brief summary of the event. Hey, it’s the least I could do.

Though I tried, I was not able to attend every session, so this is far from a complete report. That said, here were a few major themes I noticed throughout the week:

Mobile Is Mainstream

The subject of mobile app testing is getting more and more recognition in the testing world. A sign of the times, to be sure. I started the week by attending a presentation by Rama Krishna Pagadala, the Software Design Engineer in Test for Microsoft, who spoke about the need for real-world testing of mobile applications. He covered many of the same topics addressed on this blog (and mobileapptesting.com), including the unique factors that make mobile a much greater testing challenge. Specifically, variables such as low battery, connection speed, continuously running apps, OS, screen size, user preferences and others.

Rama talked a bit about the performance testing he did for the Microsoft Office Communicator Mobile application for Windows Mobile 6.x phones, including one set of automated tests that measured six performance metrics for 20 different scenarios. Fascinating stuff.

Then there was Karen N. Johnson, consultant and author of Beautiful Testing, who’s second session dealt with Functional Mobile Testing. Here, Johnson used a great series of screen-shots to show how simple web tasks like sign-in, log-out and others are exceedingly complicated when it comes to mobile. She was also very careful to stress the importance of manual testing over automated options, saying something to the effect of, “if you don’t know what you want tested, how are you going to automate it?” Good question, right?

Continue Reading

Essential Guide to Mobile App Testing

uTest @ CTIA Wireless! [pics, news, videos and more]

It turns out that launching a service as monumental as uTest Express from the show floor at CTIA Wireless is a good move! It garnered incredible national and international media attention — from TechCrunch (and TC EU), ReadWriteWeb (and RWW Spain) and The Marker Israel to leading mobile publications WirelessWeek and RCRWireless to Boston’s top local blog BostInnovation.

Here’s a peek at the RCR interview with our CMO, Matt Johnston:

Of course we weren’t the only ones launching new products and services from sunny Orlando, FL.

Let’s start with new smartphones (yes, even waterproof smartphones!). Much like CES and Mobile World Congress, CTIA was fertile ground for the release of cool 3D phones such as the LG Thrill 4G and the HTC Evo 3D, and T-Mobile’s G2X which was also named CNET’s Best Phone.

And we can’t forget the new tablets. Samsung released the “world’s thinnest tablets”! The Samsung Galaxy 8.9 and 10.1 tablets are 8.6 millimeters thick, thinner than the iPad 2 (8.8 millimeters), earning them CNET’s Best Tablet. Check out the other 8 tablets revealed at CTIA and reviewed by PC World here.

If you weren’t able to make it this year, here are some pics from the show floor. I was personally moved by NTT Docomo Japan’s empty 40×40 space this year. As FierceWireless put it, “a bare floor decorated with a simple vase of the white, cherry blossom flowers with a call for funds to support the victims, was a pitch-perfect show of support and good taste.” (Pics after the bump.)

Continue Reading

Essential Guide to Mobile App Testing

A Poor Mobile User Experience Could Make You “Sick” [case study]

We all know a mobile app released with a poor user experience pretty much guarantees its failure in the app stores. NHS Direct, an extension of the UK’s National Health Service, could not afford to take that risk (after all, there were patients on the line, literally), and turned to uTest for usability testing before their app hit the Android Market and Apple App Store.

Originally designed to be used via landline phones, NHS Direct – a service giving patients the ability to assess symptoms, get self-care and advice from nurses on call – eventually migrated to the web, and most recently, to smartphones.

Charlie Young was responsible for overseeing testing of the the app across iOS and Android OS devices. Lacking the in-house resources needed for this type of project, Charlie turned his attention to uTest.

“The safety of the application wasn’t an issue at that point,” said Charlie, who mentioned that the clinical requirements had already been met. “What we really needed to know was how humans would use this app. People use smartphones in many different ways, so we needed to make sure the user experience lived up to expectations – and that’s where uTest provided such value.”

Continue Reading

Essential Guide to Mobile App Testing

15 Bugs Away from Drupal 8

Back in 2009 when we relaunched the uTest website (who here remembers our old logo?), we chose Drupal 6 for our home page CMS. Back then, I was certain that Drupal 7 would launch “real soon now” and that we would be thinking about an upgrade pathway sometime in early 2010.

My prediction was a bit off (Drupal 7 actually launched in January, 2011). That’s not a bad thing because good software takes time, and our early evaluations of Drupal 7 have been very positive. We’re excited to begin migrating as soon as we can. Yet given that Drupal 7 took three years to develop, one has to wonder if we’ll see Drupal 8 before 2014.

With that in mind, I was a bit surprised to read Drupal founder Dries Buytaert’s plan for launching Drupal 8 on a more reasonable timescale: never have more than 15 critical bugs in the database. From this recent article in eCRMGuide:

“At no point in time will there be more than 15 critical bugs,” Buytaert said. “I will not pull in a big change if we know there are known bugs. This gives us the ability to do timely releases because we know at most the release is only 15 critical bugs away from being ready.”

So wait, is this practical?

Continue Reading

Essential Guide to Mobile App Testing

Version 3.5 – Custom Reviews and Tester Device Requirements

Last month, we launched Reviews with version 3.4 of our testing platform, and now the reviews are in (pardon the pun). Customers love the new feature, and testers enjoy having a new way to deliver structured app feedback.

Custom Reviews
For version 3.5 of our platform, we’re taking Reviews and making them better. The biggest feature request has been to make it easy to add new review questions and customize the feedback process. Our engineers tackled that requirement with gusto, and today we’re launching a new way to modify reviews.

The process is very easy: you can remove the default questions, add new questions, or change the question order to reflect your priorities.

We think reviews are a great feature for any customer looking to get feedback from real people using their software. And of course, they’re entirely optional (and can be enabled as part of any test cycle). Talk to your project manager to learn more.

Tester Device Requirements
As our community has grown, we have gathered over 35,000 enthusiastic software testers who love technology. Many of them own more than one computer or mobile device, and a few of them own dozens. In the past, testers might be matched to a project because one of their devices made them a good fit, but they had no way to know which device to prioritize for testing. Customers or uTest project managers would have to reach out to each tester to let them know that they should, for example, use their iPhone and not their Android device to test a mobile website.

With this release, we’re introducing a new system for testers to help guide them to the right device for testing. Customers can specifically say that a tester should focus on testing with specific devices while disallowing others.

Have an idea for future product releases? uTest community members can join our tester forums and check out our Platform Feedback section. Customers can contact their project manager directly or drop us a line.

Essential Guide to Mobile App Testing