Software Testing Blog

Bug-iversary Alert! Tomorrow is the 20-year anniversary of the “crash” of the AT&T Long Distance Network. On January 15, 1990 faulty software was installed on the AT&T Electronic Switching System (Number 4 ESS): a one-line bug incapacitated the entire system, disabling switches throughout half the network.

Known as one of the most serious telecom bugs in history, more than 75 million calls were not connected during 9 hours, an estimated $60 million loss.

Dennis Burke of California Polytechnic said it best: “The Jan. 1990 incident showed how bugs in self-healing software can bring down healthy systems, and the difficulty of detecting obscure load- and time-dependent defects in software.”

Speaking of “load defects,” AT&T — after signing up to be exclusive U.S. provider of iPhone service — has recently come under fire for the quality of its network coverage. Businessweek‘s top headlines read:

• An AT&T Mystery: Abrupt New York iPhone Shutdown
• AT&T Mulls Plans to Deal with iPhone Data Demand
• Can AT&T Meet iPhone Network Demands?

In light of this bug-iversary, I can’t help but wonder if more testing should have been done before AT&T took on the massive data demands of modern 3G smartphones? What do you think?

Today, we announced the results of our 2009 uTester of the Year Awards. Our community is full of professional testers, which made the judging incredibly tough (I can’t believe how much the bar has been raised for testers over the course of 2009). This awards program, however, was designed to recognize those few testers whose testing skills, attention to detail and consistently excellent performance stood out.

The winners were selected by our community management team and project managers, and were based upon testers’ performance across several hundred test cycles for web, desktop and mobile applications.

Brian Rock from Austin, Texas was named the overall uTester of the Year.  Brian joined uTest early in 2009 and brings 10+ years of software engineering experience to our community. Over the course of the year, Brian earned MVT (Most Valuable Tester) awards on multiple test cycles and also wrote a popular uTest guest blog post, “Software Testers: The Eyes of the Battlefield.”  He consistently reports excellent bugs, communicates with customers extremely well, and is very engaged in uTest projects.  Brian had this to say about his experience with uTest:

“Working with uTest challenges me to learn new applications and to solve new testing problems on different products every week,” said Brian Rock. “This keeps things fresh and exciting, and opens my eyes to see systems holistically and keep my defect localization skills sharp. This is what I enjoy most about working with uTest, and I am honored to be among this elite group of testers.”

The complete list of winners of is available after the jump:
Read more…

As a way to extract the collective wisdom of the uTest community, we decided to experiment with a Guest Blogger program beginning in April. To say that it’s been a success would be an understatement, but we’ll say it anyway (the number of page views don’t lie!). Having covered a wide range of topics – including mobile app testing, tester overconfidence, security testing and more – the series has become a big hit within the community — and a great way for testers to get published in front of a large audience.

Here are a some of the highlights from our 2009 guest blogger program.  Stay tuned for an even bigger 2010!

Who is the User? – by Lucia Maldonado:  In what ways is software similar to architecture? And how can this help steer testers in the right direction? In this post, Lucia Maldonado takes an in-depth look at user accessibility standards, and offers a number of essential tips for testers in this field.

Security Testing Tips (from a Bug Battle Winner) – by Bernard Shai Lelchuck:  When it comes to security testing, few can match the expertise of Bernard Shai Lelchuck – one of uTest’s first (and finest) QA professionals. In this post, Bernard covers the basics methods of security testing, including tips for  information gathering, logical attacks and injection attacks. Oh, and here’s Part II.

Respect the Defect: Advice That Will Change the Perception of  Testing – by Joseph Ours:  Testers need to reconsider they way they report bugs – this was the position taken by Joseph Ours in his first (and hopefully not last) uTest blog post. Challenging testers to demonstrate their value by writing more clearly about the bugs they uncover (among other tactics), Joesph has sparked an interesting debate among our community. Visit the comments section to see for yourself.

Step Away from the Simulator: Putting Mobile Applications Into a Tester’s Hands – by Brad Sellick:  What makes mobile testing different from conventional software testing? For one, the simulators and emulators are far less reliable. In this post, uTester Brad Sellick – a self-made expert on mobile app testing and development – explains the dangers of relying on these tools while performing mobile app testing.

What You Need to Know About Writing Effective Test Cases – by Valerie Dale:  Despite all evidence to the contrary, test case design is often seen as work with no real value – a remedial task with no significant ROI. One would think that with the added pressures to launch a quality product on schedule, test case design and planning would be a top priority. It’s not. At best, there is minimal attention paid to the practice. At worst, it’s non-existent. In this post, Valerie Dale makes a great defense of  this beleaguered practice.

Your Overconfidence is Your Weakness: Lessons from a “Crash Specialist” – by Pradeep Soundararajan:  In our most-popular guest post to date, noted blogger Pradeep Soundararajan explains why finding lots and lots of bugs isn’t necessarily a good thing. Reliving his days as a “crash specialist” Pradeep examines how a tester’s ego can get in the way of their objective. His advice is as funny as it is useful. Simply put: a must read.

Software Testers: The “Eyes of the Battlefield” – by Brian Rock:  Our testers come from all sorts of backgrounds, including the armed forces. Brian Rock – a former Sgt. for Combat Arms Forward Recon Team in the U.S Army – is a great example. In this post, Brian makes analogizes testers with cavalry scouts. That is, they are the “eyes of the battlefield.”  Advocating exploratory software testing (especially for those in the uTest community) this post will make you rethink the role of testers.

You’re a Professional Mobile Tester (you just don’t know it yet) – by Bernard Shai Lelchuck:  As the title would imply, this post makes the case that anyone with a mobile phone and an inquisitive mind can become a successful mobile tester. It worked for Bernard Shai Lelchuck! Here Beranrd explains the rise in mobile applications, how he himself broke into the field and some basic tips for those who would like to get started in this growing (and highly lucrative) field.

Question the Connection: Tips for Diagnosing User Login Failures – by Sherry Chukpa:  Forget the sweeping generalizations about software testing “best practices.” This post by uTester Sherry Chupka gets right to the point on a very specific issue: user login failures. If you’ve ever been pitted against this problem in the testing lab, Sherry feels your pains, and has some invaluable advice for you as you move forward.

It’s been a great year, with some terrific insights into the world of testing, but our Guest Blogger program is just getting started. So if you have an opinion to express, a tip to share or a bone to pick, we’re always eager to share the thoughts of our tester community. Email us your ideas at marketing@utest.com.

Late last week, our fearless leader was out in the Valley presenting at the Under The Radar Mobility 2009 event.  We first presented at UTR in April as an emerging company.  This time, we were asked back to present as a Graduate Circle company, which is reserved for those past presenters who are making a big splash in the market — quite an honor indeed.

The theme of this event was all things mobile.  The event was heavily attended by mobile OEMs, wireless carriers and all types of mobile app makers, including innovators such as Boku, Mplayit, ShoZu and Yowza!! (those are their exclamation points, not mine).  More details (and fantastic photos) after the jump.

Read more…

When our Guest Blogger series began a few months back, you might recall that it was Bernard Lelchuck who got things started. For those who are new to uTest, Bernard has been one of our top testers from the get-go, and you can read more about his background and uTest experience by checking out his Tester Spotlight. In his latest post, he explains how he got into the lucrative field of mobile app testing – and how all testers can (and should) do the same. Enjoy!

If you haven’t noticed, the use of mobile applications has skyrocketed over the past few years. And while most mobile companies are lagging behind Apple’s success, the market itself has nevertheless become a multi-billion dollar endeavor. As one might expect, this success has prompted competitors of all sorts to rush and open their own mobile application stores. They naturally seek greater market share, and who could blame them?

According to a recent report published on the Wireless Expertise website, “the global mobile app market – including games – will be worth $4.66 billion in 2009, rising to $16.60 billion, in 2013.”

This of course would help explain the sudden entrance of Microsoft, Google, Research in Motion (RIM) and Palm, along with mobile vendors like Verizon and AT&T into the mobile market. As I like to say, they are trying to catch the fast-riding “Mobile App Train.”

And what a ride it’s been! Since the 1^st gen iPhone was released in June of 2007, almost every leading mobile vendor has changed their products to look, feel and be as cool as the iPhone (with varying degrees of success).

Which brings me to mobile testing. But before I discuss the testing implications of this iPhone mimicking trend, I’d like to address how I got into mobile testing in the first place.  It’s my hope that this story will encourage other testers to consider furthering their careers by hopping on board the Mobile App Train.

Read more…

Last week we announced that, during the first few months of ’09, we’ve observed tremendous growth in mobile application testing within the uTest marketplace.  Since then, we’ve received a surprising number of requests for more info from mobile developers and members of the media.  It seems that testing and quality is a bit of a hot topic among those who spend their days & nights working on (or writing about) mobile.

One of the most common questions is whether our QA community has coverage for a range of mobile platforms like iPhone, Android and Blackberry.  The answer to all of the above is ‘yes’ (the answer to the multi-location, multi-language question is ‘yes’ as well).

This announcement has already resulted in several great articles from top-shelf tech sites, including eWeek and TechTarget, and got reported by mainstream media outlets such as MSNBC, MarketWatch, TMC.net and StreetInsider.

Over the weekend, as I pondered why this simple announcement resonated with so many mobile app companies and journalists, two thoughts came to mind:

1.  Shut up.  Don’t overthink this — just ride this wave!

2.  Testing for mobile apps is a widespread and serious problem with few viable solutions.  In fact, getting mobile apps properly tested is even tougher than web apps because the space is more complex and less mature.

And I guess that’s where a global community of professional testers plays nicely into the mobile app equation — because the hardest thing to build (global coverage across locations, languages, handsets, carriers, operating systems, etc.) is baked right in.  And the fact that our community can test mobile apps quickly, efficiently and on-demand doesn’t hurt either.

I’d love to hear from mobile app developers about how they test today, or from testers about the challenges that are unique to mobile. It’s a topic that’s going to be top-of-mind for the next year or more (just wait ’til mobile really clashes with the enterprise), so let’s start the discussion now.