Software Testing Blog

Bug-iversary Alert! Tomorrow is the 20-year anniversary of the “crash” of the AT&T Long Distance Network. On January 15, 1990 faulty software was installed on the AT&T Electronic Switching System (Number 4 ESS): a one-line bug incapacitated the entire system, disabling switches throughout half the network.

Known as one of the most serious telecom bugs in history, more than 75 million calls were not connected during 9 hours, an estimated $60 million loss.

Dennis Burke of California Polytechnic said it best: “The Jan. 1990 incident showed how bugs in self-healing software can bring down healthy systems, and the difficulty of detecting obscure load- and time-dependent defects in software.”

Speaking of “load defects,” AT&T — after signing up to be exclusive U.S. provider of iPhone service — has recently come under fire for the quality of its network coverage. Businessweek‘s top headlines read:

• An AT&T Mystery: Abrupt New York iPhone Shutdown
• AT&T Mulls Plans to Deal with iPhone Data Demand
• Can AT&T Meet iPhone Network Demands?

In light of this bug-iversary, I can’t help but wonder if more testing should have been done before AT&T took on the massive data demands of modern 3G smartphones? What do you think?

uTest was on fire at Under the Radar Mobility this year. I think Under the Radar said it best!

For anyone looking to deploy an app across multiple mobile platforms and a gazillion different handsets, one massive problem awaits them: QA. uTest solves this problem with an army of testers across the world. Crowdsourced QA… Problem solved. (Click here to see Doron’s presentation.)

And that’s not all! Doron was able to connect with multiple partners, prospects and top media outlets, including Mashable’s Ben Parr (@benparr), editor in chief at IntoMobile.com Will Park (@willpark), ReadWriteWeb’s Dana Oshiro (@suzyperplexus), as well as participated in a couple great video interviews with bnetTV’s Michelle Sklar (@bnettv) and GoMo News’ Cian O’Sullivan (@gomonews) which are posted below!

Take a peek at the video interviews below to learn more:

Doron Reuveni-CEO of uTest speaks with bnetTV.com at the Under the Radar event.

Read more…

Mobile bugs are becoming a bigger and bigger problem, and iPhone users are the latest to be affected by buggy software.  Techcrunch reported yesterday that a bug in Safari causes it to consume bandwidth even when it’s closed.  The problem arises with the use of Motion JPEG (or M-JPEG), a video standard that is built off of the JPEG images standard.  When an iPhone user visits a page with an M-JPEG video, Safari will download continuously, even if the user pushes the Home button to end Safari and go back to the home screen.

What makes this problem really terrible is that many mobile users are charged for their bandwidth usage.  Even AT&T users, who are used to having unlimited data plans in the United States, can be charged for data consumption while traveling internationally.  That can mean big bills when they get back home.

Cellular service providers aren’t immune either.  Networks around the world are struggling to keep up with the data demands of the iPhone, and providers certainly don’t need to deal with unnecessary data consumption caused by broken software.  That means companies like AT&T are building out extra infrastructure to support buggy apps – something that costs you and me money in our phone bills.

For the security researchers who uncovered this bug, one hour of testing resulted in $3,000 worth of data charges. That is serious money, and anyone developing mobile apps should take heed.  Your broken app can cost enough money to buy 10 iPhones, and that’s a great way to get some bad press.  Further proof that mobile application testing needs to catch-up to the standards of web and desktop testing.

Last night, our development team unleashed version 2.7 of the uTest platform.  This release includes some great enhancements – many of which were based on requests from our customers.  We’re especially excited about a few features in particular:

Enhanced Mobile Releases
Mobile app testing has always been something we’re proud of, but as the number of mobile testers and customers has grown, we’ve outgrown our normal release structure.  With so many mobile phones, carriers, and operating systems around the world, mobile customers have asked us to make it even easier to choose between them for new releases.

Now customers will experience an improved workflow for mobile releases.  We help you select just the right specifications for your testing – meaning you get just the right testers you need for your project.  So whether you’re looking for Blackberry testers in Britain or Android testers in Argentina, we make it easy to get started testing.

(click thumbnail for a larger view)

Niftier NDA Support
While we have long supported non-disclosure agreements between testers and customers, the process has been more cumbersome than we would have liked.  Starting today, customers can upload a non-disclosure agreement to which testers must read and agree before starting a release.  This will greatly simplify the process of using NDAs on releases and make it possible for customers with IP sensitivities to get going even faster.

Customer Notes
Many of our customers have been asking for a way to add their own notations to a bug report separate from the tester’s feedback.  Sometimes testing managers want to share information back and forth within their team, track the progress for bug verification, or capture quick thinking on bug followup.  Customer Notes are private and only visible to Testing Managers and uTest Project Managers.  Individual testers will never see them.

Customer notes are located on the right column of individual bug reports.

Other Fixes and Enhancements
We also made a number of other improvements including beefing up features like .CSV exporting, streamlining some of our back-end project management, and tuning up our platform’s performance to handle larger and larger traffic loads.

Our tester community, as always, offered some great ideas and helped us find and fix a number of bugs.  Have a great idea for our future product releases?  Join our testing forums and check out our Platform Feedback section.

Speaking of the mobile app market blowing up, I wanted to give you a heads up regarding a great mobile event: Under the Radar on 11/19 in Mountain View, CA.

From global carriers and handset manufacturers to media companies, branding partners, press and VCs — anyone who’s anyone leveraging new mobile tech and interested in catching a glimpse of the 2010 mobile marketplace should attend.

Featuring the most cutting-edge mobile startups from around the globe, Under the Radar will no doubt be another hit.

In the past 3 years, presenters at Under the Radar have gone on to raise over $1.36 Billion! Other knock-your-socks-off stats from the show:

• 49% have gone on to raise funding or be acquired
• 14% have been acquired by companies such as Google, eBay, Microsoft, Yahoo and Cisco
• $14 Million average has been raised by presenting companies.

An Under the Radar grad ourselves, we highly recommend it!

Last week we announced that, during the first few months of ’09, we’ve observed tremendous growth in mobile application testing within the uTest marketplace.  Since then, we’ve received a surprising number of requests for more info from mobile developers and members of the media.  It seems that testing and quality is a bit of a hot topic among those who spend their days & nights working on (or writing about) mobile.

One of the most common questions is whether our QA community has coverage for a range of mobile platforms like iPhone, Android and Blackberry.  The answer to all of the above is ‘yes’ (the answer to the multi-location, multi-language question is ‘yes’ as well).

This announcement has already resulted in several great articles from top-shelf tech sites, including eWeek and TechTarget, and got reported by mainstream media outlets such as MSNBC, MarketWatch, TMC.net and StreetInsider.

Over the weekend, as I pondered why this simple announcement resonated with so many mobile app companies and journalists, two thoughts came to mind:

1.  Shut up.  Don’t overthink this — just ride this wave!

2.  Testing for mobile apps is a widespread and serious problem with few viable solutions.  In fact, getting mobile apps properly tested is even tougher than web apps because the space is more complex and less mature.

And I guess that’s where a global community of professional testers plays nicely into the mobile app equation — because the hardest thing to build (global coverage across locations, languages, handsets, carriers, operating systems, etc.) is baked right in.  And the fact that our community can test mobile apps quickly, efficiently and on-demand doesn’t hurt either.

I’d love to hear from mobile app developers about how they test today, or from testers about the challenges that are unique to mobile. It’s a topic that’s going to be top-of-mind for the next year or more (just wait ’til mobile really clashes with the enterprise), so let’s start the discussion now.

Hi uTesters – Matt Johnston here.  I wanted to quickly introduce myself and let you know that I’ve joined the uTest team as VP of Marketing.  Much of my background is in building and marketing marketplaces, so the uTest concept appealed to me immediately.  And my first few weeks here have further raised my expectations for what we’re building together.

Anyone who’s been around software or web projects knows that testing is the end of the whip.  After the scope finally stops creeping; after the developers have churned out the code; after the schedule has slipped repeatedly; it’s the testing phase that often absorbs the hit.  uTest and our community enable companies to do all of the above and still release high-quality web, desktop and mobile apps on time.  That’s incredibly powerful.

We’re going to have a lot of exciting stuff coming in 2009 that will attract more great customers and testing projects, as well as continuing to build our community of talented testing professionals.  Stay tuned.

In the meantime, I’ll be posting on a regular basis, as well as joining our testers in the forums.  If you have any ideas or feedback that you want to share with me – positive or negative – I’d love to hear it.  You can drop me a comment here or email me at mattj [at] utest.com.