Software Testing Blog

After countless hours of review, discussion and debate (and a few late nights of pizza and beer), the official results of the Q3 Bug Battle are in!  During a week in late August, a total of 606 bugs were reported across Google, Google Caffeine, Bing and Yahoo; and nearly $4,000 in prize money was doled out to those testers who’s bugs and feedback were judged to be the best.  For complete details, see the Q3 Bug Battle report.

Inside uTest, we all agree that this was the best Bug Battle yet!  With more than 1,100 testers from 50+ countries around the world participating, we had more high-quality bugs and feedback than in any of our previous three Bug Battles.

First, let me say thanks to all the testers who participated.  Second,  it’s safe to say that this was the toughest Bug Battle we’ve had to judge yet (a good problem to have).  Third, I want to express our sincere congratulations to all the winners!

The top tester prize went to Joseph Ours of Columbus, Ohio, who reported a number of highly valuable bugs and provided some outstanding feedback. Other winners included noted testing blogger Pradeep Soundararajan of India (top bug), as well as Max Sterneck from Japan (top novice).  Other big winners included Bernard Lelchuck of Israel and Ranchhod Prajapati of India.  Check out the complete list of Q3 Bug Battle winners .

A Few of the High-Level Findings from the Bug Battle:

• 71% of respondents stated search results accuracy as the most important when choosing their search engine.
• In overall accuracy, Google led the way with a top two box score (those rated as “excellent” or “good”) of 90%. Caffeine followed with 83%. Yahoo & Bing trailed with 53 and 42%, respectively.
• Bing had the most reported bugs; however, it scored very high for usability and pics/vids search (note: yesterday, Bing unveiled a new “visual search feature)
• More than 30% noted that Bing surprised them favorably; and 10% said that after testing all three engines, they would make Bing their default engine.

The Final Bug Results:

• Google
  • 130 bugs reported
  • 8 percent classified as “showstoppers”
  • 28 percent were Technical, 46 percent were Functional, and 26 percent were GUI
• Bing
  • 321 bugs reported
  • 14 percent classified as “showstoppers”
  • 29 percent were Technical, 46 percent were Functional, and 25 percent  were GUI
• Yahoo
  • 70 bugs reported
  • 10 percent classified as “showstoppers” and 50 percent as “high severity”
  • 28 percent were Technical, 57 percent were Functional, and 15 percent were GUI

Don’t want the Bug Battle bliss to end?  You can get your fix by checking out the:  Press Release, Full Report, & Bug Battle Winners!  Thanks again to all our testers and get ready for our Q4 bug battle, which we’ll be rolling out later this fall.  If you have ideas for what we should test next, drop us a comment or shoot us an email!