Bug Battle Results!
Posted on December 9th, 2008 in uTest by Stanton Champion
After a lot of judging, we are very pleased to announce the results for the first ever uTest Bug Battle!
Over 1,330 uTesters from 68 countries competed to see who could find bugs in three of the top web browsers – Internet Explorer 8, Firefox 3.1 beta and Google Chrome. During their days of testing, they found a total of 672 bugs. The uTest community identified 101 of those bugs as “showstoppers,” which are defined as flaws in need of immediate attention. The final Bug Battle results included:
- 356 uTesters evaluated Internet Explorer 8 and found 168 bugs, including 9% that were deemed showstoppers
- 514 uTesters evaluated Firefox 3.1 beta and found 207 bugs, including 24% that were deemed showstoppers
- 461 uTesters evaluated Chrome and found 297 bugs, including 12% that were deemed showstoppers
Congratulations to our Bug Battle Winners!
The whole comparison does not make sense to me, as it is not a fair comparison in the first place. Firefox has source code for bughunters to scrutinise. IE should also disclose the source code for fair ground comparison.
Also, in terms of number of features, Firefox > Chrome > IE. Let all browsers have equal number of features, then we see who has more bugs. The fact that Chrome has more bugs than Firefox testify that despite that Firefox has more features (Add-ons support, private browsing, RSS, etc).
I think we’re drawing some bold conclusions here from beta testing: in the real world, no one is claiming that the beta versions are bug-free. The real question is what about released products: Firefox 3.0 vs. IE7. How many showstopper bugs are in these officially supported versions?
(Yes, I realize that Chrome was just taken out of beta, but I suspect that the testing was done while it was in beta).