Software Testing Blog

The Battle of the E-Tailers – our latest bug-hunting competition – is now one for the history books! Over the course of a week in November, a total of 505 bugs were reported in three of the world’s top online retailers: Amazon, Walmart and Target. For complete details, be sure to check out the Q4 Bug Battle Report.

While millions of shoppers were making leisurely holiday purchases, our global community of testers were hard at work – reporting a wide range of functional, usability and GUI bugs, as well as providing  user feedback. In doing so, you’ve once again demonstrated the value of comprehensive testing performed by skilled professionals. Well done!

The media, by the way, are all over this story, so we’ll be sure to provide a wrap-up of all the news coverage later this week.

To the Victors Go the Spoils
The prize for Top Tester went to Brad Sellick of Canada, while Tuyen Tran of Vietnam took home the new Judge’s Choice Award. Other cash-prize winners included Gaurang Joshi and Ranchhod Prajapati (both of India) for Best Bugs, as well as Gizem Sahin (Turkey) and Jaya Dinu (US) in the Best Feedback category. There were over 30 winners in this quarter’s competition, so be sure to check out the results page for the complete list.

And now, a few more top-line findings:

*Amazon ranked highest in most categories (pricing, ease of use, product reviews), while Walmart ranked highest in product search tools and also earned good marks for pricing.

*Target.com ranked lowest in all five categories and accounted for 261 bugs, more than half of all bugs reported in the competition.

*67% of testers did not choose “lowest prices” when asked to identify their most important category. In fact, only 1 in 3 chose lowest prices as their primary decision making criteria.

*Nearly 75% of all survey respondents said they trust Amazon most for their holiday shopping.

The Final Bug Results:

Amazon.com:

• 94 bugs reported
• 11% classified as “showstoppers”
• 21% Technical, 67% Functional and 12% GUI

Walmart.com:

• 150 bugs reported
• 9% classified as “showstoppers”
• 23% Technical, 54% Functional and 23% GUI

Target.com:

• 261 bugs reported
• 7 percent classified as “showstoppers”
• 20% Technical, 56% Functional and 25% GUI

Thanks again to all of the testers who participated in this Bug Battle. Remember, we’re just getting started with this competition, and so if you have ideas for what we should test next, drop us a comment or shoot us an email!