Software Testing Blog

Safe to say that mobile app development has greatly outpaced mobile app testing over the last few years. In other words, while the applications and platforms have seen tremendous technological advances (iPhone 4 bugs notwithstanding) the same cannot be said of mobile testing methodologies.

Case in point: The majority of mobile app developers remain overwhelmingly reliant on internal beta testing.

Here with proof is VisionMobile, who recently published a fascinating report on the growing mobile app ecosystem – a must-read for anyone involved in the space (developers, marketers, users, etc). From a QA point of view, the report further establishes that although testing innovations will ALWAYS trail those of development, the gap need not be so wide.

Here’s an excerpt that sums the whole thing up:

Internal beta testing is the most popular technique used by the vast majority (nearly 70 percent) of respondents, with beta testing with users and peer reviewing the next most popular techniques. Only 20 percent of respondents use focus groups or research of their own. Overall, North American developers are somewhat more sophisticated in their application planning, with 97 percent using beta testing as a standard part of application development and with broader use of a portfolio of planning techniques as well.

Yet, small development firms have limited means today to beta test and peer review their applications with a crosssection of representative users. Given the hundreds of thousands of mobile apps, we believe that efficient (crowd-sourced) testing of apps in a global market of users is considerably under-utilized. This presents an opportunity for the few solution providers in this segment – Mob4Hire and uTest.com, for example – but also for network operators, who can generate a channel for testing applications with end users, and provide an open feedback support system back to developers.

Other notable findings included:

Read more…

So maybe it wasn’t AT&T’s fault after all.

Apple recently revealed that there is a fundamental flaw in their method for calculating how many signal bars to display.  And we have the iPhone 4 (and its “learn to hold your phone the right way” fiasco) to thank for bringing this software snafu to light.

CNN Money shares the following details from Apple:

“Upon investigation, we were stunned to find that the formula we use to calculate how many bars of signal strength to display is totally wrong,” Apple wrote in a statement posted on its website. “Our formula, in many instances, mistakenly displays 2 more bars than it should for a given signal strength.”

That means, for example, that iPhones sometimes display four bars when they should be displaying two. Apple said users reporting a significant drop in bars when they hold their iPhone 4 are probably in an area of “very weak signal strength” but were unaware of that because the phone displayed four to five bars.

“Their big drop in bars is because their high bars were never real in the first place,” the company said.

Perhaps most surprising, Apple disclosed that the problem is not confined to the iPhone 4.  The faulty formula has been present in every iPhone model since the 2007 original.  Questions remain about whether the issue is strictly software-related, or if it also involved hardware problems.  However, Apple has said it will release a free software update in the next several weeks to fix the glitch. It will use a new formula recommended by AT&T.

Almost like clockwork, Apple has consistently announced iPhone updates at their WWDC conference every June.  This year was no exception with Steve Jobs himself presenting the much awaited iPhone 4. Just as in the past, this year’s update introduces a number of interesting and must-have features for mobile developers and aficionados everywhere. Here are some of our favorites:

New Display – a Treat for Sore Eyes
Before today, I could not have told you the maximum amount of detail my retina could process in a given inch of space.  Now, thanks to Steve Jobs, I know: 300 dots per inch. That’s important because the iPhone 4 features an astonishing 960×640 display that resolves to 326 pixels per inch. That’s apparently more than what your retina can physically see, a fact that Apple took to heart when they decided to name this feature “Retina Display“.

More incredible is how this compares to the iPad. The iPad is a significantly larger device, but with a resolution of 1024×768 it only has 28% more pixels.

Of course, this also means that Apple developers now have three resolutions to worry about when they’re developing apps: iPhone 4, iPad, and iPhone/iPod 2G/3G/3GS.

Read more…

If you live in the United States and own and iPhone, chances are you’re aware that AT&T has some reliability issues.  As the sole mobile service provider for iPhones in the US, AT&T has seen their usage skyrocket and their reliability crumble in the past few years. Everyone has an opinion about why, but as of today AT&T has revealed what they see as one big problem: heavy bandwidth usage.

Starting this month, AT&T will now limit the amount of data an iPhone or iPad may consume in a given month.  The caps start at 200MB, but for an additional fee you can have all the way up to 2GB.  Existing users will continue to have an unlimited data plan for now, but new contracts will not have that option going forward.

AT&T says that 98% of their customers use less than 2GB of data per month, which means that most people won’t hit the top usage cap.  However, many people will still opt for the lower 200MB data cap because it will actually be cheaper than their current unlimited iPhone data plan.  That means software developers have an interesting new challenge on their hands – bandwidth optimization.

Read more…

As the smartphone battle heats up, a debate has begun around a seemingly crucial question: which platform is faster?  In a lot of ways, that’s impossible to answer. Performance comparisons depend on many factors, including the tradeoff between performance and battery life.  But that hasn’t stopped some from having the debate anyway, and the battle lines right now are over Android’s JVM vs. the iPhone’s Objective C objc_msgSend().  Let me explain.

Android is a Java based platform and uses a Java Virtual Machine or JVM to execute apps (Android’s JVM is called Dalvik). Historically, Java was considered to be a slow and cumbersome platform.  The joke was write once, run anywhere very, very slowly.

So people are saying that Android is slow, right?  Actually, no.  In fact, Java has been optimized so extensively in the past 10 years that its performance is now incredibly fast.  Dalvik has been optimized even further for mobile devices, so Android is one fast platform. What people are actually saying is that the iPhone is slow.

Read more…

Worldwide productivity surely took a nose-dive today, as thousands of worker bees (like me) discovered that Google was featuring the classic Pac-Man arcade game on its ever-changing homepage. What began as a scholarly search for “regression testing tips” quickly devolved into “five” minutes of ghost-chomping fun – but don’t tell my boss.

In honor of Pac-Man’s 30th birthday, Google developed the application (in what we presume is HTML5) to look, sound and behave just like the original version from 1980. [UPDATE: Here's how to download the Pac-Man game for free] We’re not yet sure if this includes the infamous Pac-Man kill-screen bug, but I am determined to find out. I’ll work nights and weekends if that’s what it takes. That’s just the kind of dedicated employee I am.

Anyway, since we’re a software testing company, many of us in the office were curious to see how the application would perform on the various mobile devices we have in-house. Here’s a quick run-down of our findings for each device, including whether or not it worked, along with a few notes:

Read more…

Like a rocket ship breaking the bounds of Earth’s gravity… like a bird soaring majestically over the open sunlit plains… like a spit wad hurled from the back of the classroom… today, uTest announced the launch of MobileAppTesting.com.  You can also follow our wit and wisdom on Twitter @mobile_app_test.

One thing that you won’t find on this site are ads, subscriptions, hooks or any kind of commercial agenda (don’t tell our investors!).  In fact, we created this site simply because mobile is the next frontier of app development and testing – and the fastest-growing segment of uTest’s business. So we wanted to give something back to mobile app developers, testers and entrepreneurs — and have a little fun at the same time.

We’ll will work with partners, pundits and pioneers (actively seeking co-conspirators) to create original, thought-provoking content about the entire mobile app ecosystem — from app developers to device makers to wireless carriers.  Whether it’s the apps arms race, the constant carrier battles, or the next must-have device, MobileAppTesting.com will be there with equal parts education and entertainment.

This site features user-generated content, contests, product reviews and guest interviews with mobile execs… stuff you can’t find anywhere else, including:

Hard-earned lessons for entrepreneurs, techies and investors who want to create world-class mobile apps

Overcoming obstacles unique to developing and testing apps for iPhone, Android, BlackBerry and Symbian

Breaking news and product reviews (for apps, devices, networks and more) from top bloggers and journalists on the front lines of the mobile app explosion

Interviews from people who live it, offering their insights from the worlds of mobile app marketing, design, development and testing

Following uTest to mobile industry conferences, networking events, speaking opps and meet-ups

Want to be published on MobileAppTesting?  Have a topic you want us to tackle?  Feel the need to ask what the heck we think we’re doing?  Shoot us a note or drop a comment.

Just hours ago, Apple CEO Steve Jobs (we’ve heard of him) announced several exciting changes to the iPhone platform. Dubbed iPhone OS 4, this update will bring some awesome new features to iPhone owners sometime this summer. You can catch up on all of the changes here, but three four of them are super important for iPhone app developers and testers.

1. Multitasking
Implementing multitasking on the iPhone has always been a challenge. Having too many apps running at the same time can reduce battery life, and managing several apps at once can be a drag on such a small device. But there has always been a need for some apps to stick around and keep running while the user is doing other things. Apple is taking care of this by offering multitasking “services” – seven of them in all.

Starting with OS 4, you will be able to take advantage of rudimentary multitasking for features like audio, VOIP, and location. The new OS will also support “freezing” your app when the user switches away and then restoring it when the user switches back. Finally, a new service will help your app finish up time intensive tasks while the user moves along to something else (the example given was finishing up a photo upload).

Read more…

If media coverage equaled market share, then I’d be writing this post from my iPhone (I’m not) and every single one of you would be reading it from your shiny new iPad (you’re not).  In case you haven’t been near a TV… or a computer… or a radio… or people… you’re aware that Apple launched a new product last week called the iPad.

And with the apparent ubiquity of the iPhone, one can only assume that Apple’s mobile market share hovers somewhere between 97% and 109%.  Unless, of course, you look at those pesky “statistics”, which is exactly what the fine folks at Comscore do each month. As  Jason Kincaid (@jasonkincaid) discussed recently, the latest mobile market share stats might surprise you:

Read more…

Fifteen years ago, Microsoft introduced one of their most perplexing products – Microsoft Bob.  Today Bob is known mostly as a product failure, but in 1995 it was hailed as the future of PC user interface design.  Released a few months before Windows 95 (which really was the future of PC user interface design), Bob was an audacious alternate interface built for Windows 3.1 that visualized programs on a computer as things found inside a home.

While Bob now looks like a bizarre alternate reality for UI design, in 1995 Microsoft designed Bob to help novice computer users better understand how to use their computer.  Perhaps the best known elements from Bob were the ever-present cartoon character “guides.”  Bob’s interface was primarily wizard driven, and the cartoon guides made the wizard process friendly and inviting. Check out this Youtube video to see a typical Bob session from a user point of view.

Bob flopped not long after it was launched for a number of reasons, including the fact that it was quickly overshadowed by Windows 95.  But Microsoft persevered and kept around the idea of animated guides.  We still celebrate Bob’s offspring: Clippy from Microsoft Office and Rover from Windows XP.  Oh wait, I mean the opposite of celebrate…denigrate.  Yeah, Bob’s memory is now pretty tarnished, but I think that’s short-sighted.

Today I come to truly celebrate Bob.  Keep reading for five things Microsoft Bob got right.

Read more…