Software Testing Blog

The mere thought of air travel during the holidays is annoying enough to send most people running to their nearest bus or train station.  The crowds, the lines, the delays, the zip-lock bags and 3 oz bottles of shampoo… but wait, there’s more!

Late last week, a five-hour computer glitch caused flight delays across the U.S. that were still rippling through the transportation system for most of the day.  The problem was made worse by the fact that the National Airspace Data Interchange Network failed at both its locations — Atlanta and Salt Lake City.  (Ed. note:  I’ll try hard to avoid using the word “crash” in this post.)

Bloomberg.com had this to say:

The Federal Aviation Administration blamed a four-hour software failure for causing airline delays and cancellations across the U.S.  The shutdown lasted from 5 a.m. to 9 a.m. ET after “a software configuration” malfunction today in Salt Lake City.

And The New York Times chimed in with this little bit of sunny news:

Read more…

Mobile bugs are becoming a bigger and bigger problem, and iPhone users are the latest to be affected by buggy software.  Techcrunch reported yesterday that a bug in Safari causes it to consume bandwidth even when it’s closed.  The problem arises with the use of Motion JPEG (or M-JPEG), a video standard that is built off of the JPEG images standard.  When an iPhone user visits a page with an M-JPEG video, Safari will download continuously, even if the user pushes the Home button to end Safari and go back to the home screen.

What makes this problem really terrible is that many mobile users are charged for their bandwidth usage.  Even AT&T users, who are used to having unlimited data plans in the United States, can be charged for data consumption while traveling internationally.  That can mean big bills when they get back home.

Cellular service providers aren’t immune either.  Networks around the world are struggling to keep up with the data demands of the iPhone, and providers certainly don’t need to deal with unnecessary data consumption caused by broken software.  That means companies like AT&T are building out extra infrastructure to support buggy apps – something that costs you and me money in our phone bills.

For the security researchers who uncovered this bug, one hour of testing resulted in $3,000 worth of data charges. That is serious money, and anyone developing mobile apps should take heed.  Your broken app can cost enough money to buy 10 iPhones, and that’s a great way to get some bad press.  Further proof that mobile application testing needs to catch-up to the standards of web and desktop testing.

Like many Americans, I’ve been watching housing prices very intently.  While looking at data on the website for the Massachusetts Association of Realtors, I came across a very common and ordinary web bug: a broken link.

(Click the thumbnail for a full sized version.)

Now broken links are incredibly common, and anyone with a website has had one happen to them at some point.  But I want to dig a little deeper.  Take a look at that screen shot and think about how many bugs do you see?

By my count, I see three very real bugs:

• There’s the broken link.  Looking at this error, I think this is a convoluted way of saying that the file I wanted wasn’t found.  Whoever uploaded the file didn’t set a folder correctly.
• The Folder is not set exception didn’t get handled.  Ideally, some kind of exception handler should have caught this exception and provided the user with a more coherent error message.
• The user was shown an incoherent error message.  An end user should never see this kind of error message in the first place.  It reveals information about the underlying system configuration that could be dangerous.  At the same time the user doesn’t receive any kind of friendly help about why the error occurred or how to resolve it.

Each of these is a very real bug, but each one is also more difficult to fix than the previous.  For example, fixing a broken link may be as simple as changing a file location in the content management system.  The person responsible for posting the content could take care of that.

Correcting the next bug requires help from the developers to write a proper exception handler to catch this exception. While fixing a broken link is something that could be handled in the field, writing a new exception handler usually has to be done during the development process.  This, of course, requires a solid QA process to be in place during development.

For most projects, the last bug is the most challenging to fix.  It usually requires careful planning from the earliest stages by product management, development, and QA to agree that reducing the complexity of bugs is both important and feasible.  Software testers will have to be involved early in the process so that they can understand how to discover and interpret bugs that are designed to be customer friendly or even invisible.

Even simple bugs, like broken links, can lead to deeper questions for testing and development teams.  Not every bug has to be fixed, but make sure to keep your customers and users in mind when planning your project.  Good planning could turn a boring broken link into something that’s both apologetic and helpful.

Our Bug Battle of the Social Networks is now open and running.  Starting on Wed, Feb 11th, uTesters can test the quality, features and usability of MySpace, LinkedIn, and Facebook.  The top testers will receive prizes ranging from $200 to $1,000 for showing us the best bugs and feedback — more than $3,500 in total prize money!

Visit our Bug Battle homepage for more details.  Also, make sure you read our Tips to Win the Bug Battle.

So if you’re a uTester, get started today.  And if you’re not yet a uTester, join our community and take part in the fun.  Good luck!

In Q4, we held our first-ever Bug Battle — the “Battle of the Browsers”.  This QA contest yielded 672 bugs within IE, Firefox, and Chrome.  1,300+ testers participated and uTest awarded nearly $3,000 in prize money.  Now it’s time for our next Bug Battle challenge.

After a great deal of discussion with bloggers, analysts and members of our testing community, we’ve selected a fun and high-profile “battleground”:  Social networking sites.

Join other uTesters from around the world in testing the application quality and usability of the top three social networks:  Facebook, LinkedIn and MySpace.

The contest begins Wednesday, Feb 11th and runs through Tuesday, Feb 17th.  uTest will once again be awarding $3,000 in prize money for:

• Top overall tester (based on quantity and quality of bugs and feedback)
• Top novice tester (same criteria)
• Top individual bugs in each of the sites (highest profile, most interesting, most severe)
• Best feedback (about the feature set, functionality and usability)

See complete details for the uTest Social Networking Bug Battle.  Get ready to hunt some bugs, make some serious money, and be recognized as one of the top testers in the community!

Have questions or ideas for future Bug Battles?  Drop us a comment.

I wanted to post a quick note to share a milestone in the uTest marketplace — we recently surpassed the 10,000 mark in total bugs reported.  For those of you keeping score at home, that’s 55 bugs per release, or one bug every 23 minutes since our August launch.  Pretty stunning results in a short period of time, and we’re grateful to our customers and our testing community for their support.

And the next 10,000 bugs will come even faster.  As we shared with MSNBC recently, the number of customers, releases and bugs reported doubled from Q3 to Q4.  So we’re growing rapidly and we’ll have more and more testing work for our community in the months and quarters ahead.  Thanks again.