Software Testing Blog

Pundits are predicting a breakout year for Android in 2010.  The upstart mobile OS (if you can call anything Google does an ‘upstart’) is earning rave reviews from mobile app developers, and is growing rapidly.  And while the Android Market and it’s 20,000 apps have a ways to go before catch up with iPhone and it’s 100,000 approved apps, the growth has been impressive.  As we’ve seen time and time again, he who wins the hearts & minds of the development community, wins the battle.

Also helping Android’s case are some great new smartphones from manufacturers like HTC, Motorola and Samsung.

Robin Waulters from TechCrunch wrote a great piece today detailing the milestone and making some predictions for 2010:

2010 is going to be a big year for the Android operating system, with many new handsets finding their way to stores around the world (including Google’s own phone) and an increasing number of developers building tools, games and the likes for the fast-growing platform.

One way of noticing that the OS is poised for a big breakthrough at the expense of Windows Mobile, Symbian and other operating systems designed to run on various mobile devices, is the number of applications already available for download in the platform’s own application store, Android Market.

Waulters cited recently published data from AndroidLib, which showed an wildly impressive growth line for the Android Market (see the chart after the jump):

Read more…

Late last week, our fearless leader was out in the Valley presenting at the Under The Radar Mobility 2009 event.  We first presented at UTR in April as an emerging company.  This time, we were asked back to present as a Graduate Circle company, which is reserved for those past presenters who are making a big splash in the market — quite an honor indeed.

The theme of this event was all things mobile.  The event was heavily attended by mobile OEMs, wireless carriers and all types of mobile app makers, including innovators such as Boku, Mplayit, ShoZu and Yowza!! (those are their exclamation points, not mine).  More details (and fantastic photos) after the jump.

Read more…

When our Guest Blogger series began a few months back, you might recall that it was Bernard Lelchuck who got things started. For those who are new to uTest, Bernard has been one of our top testers from the get-go, and you can read more about his background and uTest experience by checking out his Tester Spotlight. In his latest post, he explains how he got into the lucrative field of mobile app testing – and how all testers can (and should) do the same. Enjoy!

If you haven’t noticed, the use of mobile applications has skyrocketed over the past few years. And while most mobile companies are lagging behind Apple’s success, the market itself has nevertheless become a multi-billion dollar endeavor. As one might expect, this success has prompted competitors of all sorts to rush and open their own mobile application stores. They naturally seek greater market share, and who could blame them?

According to a recent report published on the Wireless Expertise website, “the global mobile app market – including games – will be worth $4.66 billion in 2009, rising to $16.60 billion, in 2013.”

This of course would help explain the sudden entrance of Microsoft, Google, Research in Motion (RIM) and Palm, along with mobile vendors like Verizon and AT&T into the mobile market. As I like to say, they are trying to catch the fast-riding “Mobile App Train.”

And what a ride it’s been! Since the 1^st gen iPhone was released in June of 2007, almost every leading mobile vendor has changed their products to look, feel and be as cool as the iPhone (with varying degrees of success).

Which brings me to mobile testing. But before I discuss the testing implications of this iPhone mimicking trend, I’d like to address how I got into mobile testing in the first place.  It’s my hope that this story will encourage other testers to consider furthering their careers by hopping on board the Mobile App Train.

Read more…

As promised, Google has made the slides and video presentations from GTAC 2009 (Google Test Automation Conference) available on the GTAC website and on YouTube. This year’s GTAC was a huge success! The theme was “Testing for the Web,” and now anyone can watch these leading thinkers discuss test automation strategies, tools, and the challenges desktop and mobile environments present when creating web apps.

Doron was among a select group of speakers chosen to present at GTAC, including Microsoft, smartFOCUS Digital, Sauce Labs and of course Google, where he examined the complimentary role a community of professional testers plays in mobile testing.

Check out Doron’s presentation below! All other presentations can now be seen on YouTube.

Mobile bugs are becoming a bigger and bigger problem, and iPhone users are the latest to be affected by buggy software.  Techcrunch reported yesterday that a bug in Safari causes it to consume bandwidth even when it’s closed.  The problem arises with the use of Motion JPEG (or M-JPEG), a video standard that is built off of the JPEG images standard.  When an iPhone user visits a page with an M-JPEG video, Safari will download continuously, even if the user pushes the Home button to end Safari and go back to the home screen.

What makes this problem really terrible is that many mobile users are charged for their bandwidth usage.  Even AT&T users, who are used to having unlimited data plans in the United States, can be charged for data consumption while traveling internationally.  That can mean big bills when they get back home.

Cellular service providers aren’t immune either.  Networks around the world are struggling to keep up with the data demands of the iPhone, and providers certainly don’t need to deal with unnecessary data consumption caused by broken software.  That means companies like AT&T are building out extra infrastructure to support buggy apps – something that costs you and me money in our phone bills.

For the security researchers who uncovered this bug, one hour of testing resulted in $3,000 worth of data charges. That is serious money, and anyone developing mobile apps should take heed.  Your broken app can cost enough money to buy 10 iPhones, and that’s a great way to get some bad press.  Further proof that mobile application testing needs to catch-up to the standards of web and desktop testing.

Mobile is the next great frontier for games, music, media and more.  In fact, Gartner says that  more than 139 million smartphones were sold last year.  And the phone makers aren’t slowing down:  Apple is planning to launch the iPhone into the Chinese market and a more affordable Android handset expected to hit the market by the end of ’09.  But haven’t we all learned that big markets make big targets — for VC dollars, for advertisers, for media coverage… and for hackers.

Doug Gross wrote a very interesting article over at CNN Tech about the one trend that could threaten to derail the otherwise unstoppable mobile movement.

Security analysts say they’ve already seen all of the major online threats — Trojan horses, viruses, worms — spreading on smartphones, often through e-mail attachments sent to the phones.

And as mobile apps have evolved from games and tip to calculators to company email, financial transactions and other mission-critical activities, the risk of security exploits grows considerably for mobile users and their employers.

The good news, experts say, is that phones present problems for hackers and other bad guys that traditional computers don’t.

Most viruses and other malware are designed for Windows, because that’s the most popular operating system. Since smartphones use a variety of different platforms, someone writing malicious software needs to pick and choose.

Wow, getting code to work across mobile platforms is so hard that even the hackers are having trouble!

Read more…

Do you own a Sidekick mobile phone (AKA the Danger Hiptop)?  Then please accept my condolences while I describe the pain and suffering you’ve experienced over the past few days.

The Sidekick is made by Danger, a company acquired by Microsoft in 2008.  As one of T-Mobile’s flagship mobile phones, the Sidekick was one of the first and most popular consumer smartphones.  Featuring a real keyboard, it offered an instant messaging application at a time when many phones were still figuring out SMS.  For IM and SMS addicts, the Sidekick was THE phone to own.

One of the Sidekick’s key features was that it kept all of your important stuff “in the cloud.”  That meant it stored all of your contacts, messages, photos, and just about everything else on a server managed by Danger.  This made it easy to recover your data in case your phone lost power or failed.  What nobody anticipated was the cloud server itself failing.

In what the BBC calls “the biggest disaster yet for the whole concept of cloud computing,” that very thing happened this past weekend.  A failed upgrade to the server managing the data for all Sidekicks resulted in the loss of everyone’s data at once.  Microsoft is now warning Sidekick owners not to turn off their devices, thus permanently deleting what little data they might have cached locally.

Anyone building an app on the cloud should be worried,  because what happened to Microsoft could just as easily happen to you too.  With that in mind, here are a few lessons for cloud computing app developers:

Read more…

By now, it’s painfully obvious to all of us that the market for mobile apps is BOOMING and shows no signs of slowing down.  But I was still amazed by the stats/news that back up the hype. Within the past few months, GigaOm has reported the following mind-blowing proof points:

• Apple users downloaded 2 billion apps
• Android’s Market will come pre-loaded on Verizon phones
• Microsoft launched its Windows Marketplace for Mobile
• Apple will hit shelves in China, the world’s largest market
• Research In Motion (RIM) expanded with its new App World
• Palm introduced premium apps for its webOS
• Verizon is opening its own app store later this year

And the list goes on and on…

Read more…

Catching up on my reading and came across an interesting article on the App Store from Jason Kincaid over at TechCrunch.  Ostensibly, this piece is about Apple’s overly strict and seemingly random App Store approval process and a high-pressure NSFW rant from a high-profile developer, Joe Stump, whose popular app has a new version that is on the outside of the app store, looking in.

Stump outlines a problem that he had with Chess Wars, the Facebook Connect-enabled chess game that came out in July. After catching a show-stopping bug soon after the initial release, his company Blunder Move promptly issued an update. Soon thereafter they noticed another bug, which they quickly released a fix for. Unfortunately, this second update has sat in App Store purgatory for many weeks now, and Apple has gone silent on when it will be approved.

What struck me — other than the string of bad press that Apple has earned via the lousy treatment of its developers — is that THIS is a new cost of bugs.

Not to lay blame at Stemp’s feet.  We, of all people, know that bugs happen.  Plus he mentions utilizing 50 beta testers and 200 unit tests, so they’re doing more testing than many.  But IF these showstopper bugs had been caught in the initial version or even in the 2nd version of Chess Wars, then Chess Wars wouldn’t have its nose pressed up against the window of the App Store, waiting for the powers-that-be to bless the new version.

Chalk it up as another real-world reason to achieve maximum professional testing coverage to eradicate all quality, security and privacy defects before your app goes live.  Have other stories of software bugs causing havoc in interesting, scary or funny ways?  Drop us a comment and tell us about it.

With the summer sun finally paying us a visit in Boston, so too is the mobile testing space heating up out here! Tomorrow (7/16), Doron Reuveni will be speaking at the 2009 Software Industry Conference (SIC) at the Boston Marriott Quincy on how crowdsourcing is helping companies test their mobile apps.

With the new iPhone 3G S, Blackberry Storm 2, and the new Palm Pre, the smartphone apps competition is fiercer than ever! Mobile app developers are battling for media attention, consumer dollars, VC investments and more.

But, status quo testing doesn’t quite work for mobile apps. Multiple testing criteria — such as wireless carriers, handset makers and models, locations, languages, platforms — all must be taken into account. With crowdsourcing, any mobile app company can have world-class QA coverage.

In his session, Doron will examine the growing trend of crowdsourced software testing for mobile apps – highlighting its benefits, challenges and uses as compared with conventional QA options.

If you’re in town and want to meet up, shoot us a note!