Software Testing Blog

Hello uTesters – allow me to introduce myself.  My name is Peter Shih and I recently joined uTest as the Community Manager. My background has been in software product management and engineering, where I worked closely with the QA team and performed the fun task of testing during the critical release phases.  I appreciate the passion and skill required to find high-quality bugs in a methodical fashion, so you can imagine how excited I am to be a part of this community.

My primary responsibility at uTest will be to help serve you, the testers.  As I watch the number of QA professionals increase each day, I’ve become quite optimistic about the power and the potential of our global community.  With more and more releases coming in, there’s a growing demand for top-notch testers in all segments, particularly web and mobile applications. Therefore, I’ll be looking for ways to help testers further sharpen their QA skills, get more work, and of course, make more money through uTest.

Feel free to contact me directly at peters@utest.com with any comments or suggestions about the community, the forums, fresh ideas, etc.  Once again, I’m delighted to be joining the uTest team and I look forward to working with you!

Yesterday’s post — announcing that we need more mobile app testers to keep up with rapidly growing demand — has drawn a great deal of interest from testers around the world (which, I suppose was kinda’ the point).  One of the questions that I’ve heard from a bunch of tester in the past 24 hours is “where do you have the most mobile testing demand?”.

The answer is that, while our mobile growth has been across a variety of handset makers, wireless carriers and geographies, our greatest areas of need are:

• iPhone app testers in the US and Canada
• Blackberry testers in Spain, France, Italy and Germany

So, for the dozens of mobile testers who signed up for the uTest marketplace in the last 24 hours, thank you and welcome aboard!  For anyone else who’s interested, just send us an email and tell us your location, as well as your handset maker(s) wireless carrier(s).

I know… these two words aren’t seen very often in today’s world.

As a marketplace, uTest builds supply (testers) and demand (customers) for software testing services.  And since we have 15,000+ testing pros from 152 countries around the world, we usually have more than enough supply.  But as uTest grows, we do run into scenarios where demand grows faster than supply.

This is exactly what has happened in mobile during 2009.  In the past three months, dozens of companies have come to uTest to get their mobile apps tested.  And, while we have a lot of mobile testers around the globe, we need more.  So consider this a very public announcement that uTest has mobile app testing work available and we are actively looking for more testers to keep up with demand.

If you are a testing pro (or if you know someone who is) who wants to make some extra money for testing iPhone, Blackberry and Android apps, sign up as a tester today.  For our current crop of QA pros, update your profile to tell us which mobile devices you own.

In many cases, mobile testing offers bigger payout than web or desktop app testing.  For example, a recent Blackberry app release is paying a guaranteed $100 for a completed test script, plus $20 for each approved bug.  While payment varies by release, the point is a tester can make good money in just a few hours of heads-down testing.

Currently, our most urgent need for mobile testers is in Europe, but with the way demand has been growing, we are soliciting mobile testers in the US, Canada, South America and Australia as well.

If you know of QA pros or developers who are willing and able to test mobile apps, send them a link to this post.  Likewise, if you know of websites, message boards, forums or e-newsletters where uTest can advertise to find more mobile QA professionals, drop me a comment.

Apologies for the alliteration, but things have been ultra-busy around the halls of uTest this week.  The reason our phones have been ringing off the hook is because we announced the results of our 1st Quarter Bug Battle on Tuesday.

In case you missed it, we’ve received some fantastic media coverage in the past few days.  Who knew the world would be so interested in the results of 1,000+ QA professionals from 64 countries simultaneously testing the world’s three most popular networking sites?  Ok, well we had a pretty good idea they’d care, but we’re glad it captured their attention.  Here are a few of the noteworthy articles:

- TechCrunch: uTest Bug Battle: Which Social Network Is The Buggiest?
- Dr. Dobbs: Bugs In Social Networking Software? You’re Kidding, Right?
- eWeek:  Facebook Triumphs LinkedIn, MySpace in uTest’s Bug Battle

Our community’s ability to mobilize quickly and provide real-world testing coverage is making a strong impression on a growing number of companies.  These firms are looking for ways to do more with less — and the uTest community is primed to help them meet their QA needs.  Keep up the good work!

After a great deal of scrutiny and some lively internal debates, we’re excited to announce the results of our latest Bug Battle!

A quick recap: Over the course of a week, 1,119 testers from 64 countries competed to find bugs in the three leading social networking platforms: Facebook, MySpace and LinkedIn. In all, our testing community found a total of 718 bugs, with 84 of those deemed as “showstopper” issues (i.e. bugs warranting immediate action).

Here’s a breakdown of the final numbers from the Bug Battle report (PDF):

Facebook:  416 uTesters found 243 bugs (.58 bugs/tester), including 14.40% that were deemed showstoppers by the tester
LinkedIn:  399 uTesters found 250 bugs (.63 bugs/tester), including 9.78% that were deemed  showstoppers by the tester.
MySpace:  304 uTesters found 225 bugs (.74 bugs/tester), including 10.80% that were deemed showstoppers by the tester

Here are the key findings from our Bug Battle survey:

Testers’ choice for best overall quality:
1.LinkedIn (45%)
2.Facebook (37%)
3.MySpace (17%)

Testers’ choice for best usability:
1.Facebook (39%)
2.LinkedIn (38%)
3.MySpace (17%)

Testers’ choice for best feature set:
1.Facebook (46%)
2.LinkedIn (36%)
3.MySpace (18%)

Once again, congratulations to the Bug Battle winners, and thanks to the uTest community for your participation and enthusiasm.  Your software testing prowess always impresses us and we can’t wait to see what bugs you uncover next.  Speaking of which…

We’ve got a few ideas for our 2nd quarter Bug Battle, but we’d love to hear your suggestions.  To get the ball rolling, here are a few ideas we’ve been kicking around:

• Instant messenger tools
• Web mail programs
• Top Facebook apps
• Top iPhone apps

Tomorrow we’re going to announce some very interesting findings and discoveries from the social networking Bug Battle, but today we want to highlight the winners.

Top Tester ($1,000) – Ranchhod Prajapati
Top Novice ($750) – Evan Dinsmore
Top USA Tester ($500) - Supraja Addala Prasad

LinkedIn:

Best Bug ($200) – Ranchhod Prajapati
Best Feedback ($200) – Suriyaprabha Jeevagan

MySpace:

Best Bug ($200) – Jegan Surya
Best Feedback ($200) – Supraja Addala Prasad

Facebook:

Best Bug ($200) – Evan Dinsmore
Best Feedback ($200) – Lucia Maldonado (repeat winner!)

The uTest community submitted some really great bugs and feedback for this competition and picking winners from such a great list of submissions was incredibly difficult. Our community really knows their stuff, and we were all amazed at the incredible quality and quantity of bugs that were submitted.

I want to congratulate all of the testers who participated and we hope to see you in the next Bug Battle. Until then, happy testing!

Last November, more than 500 of you tested Firefox 3.1 beta as part of our first Bug Battle, finding over 200 bugs in the process (24% of which you deemed showstoppers). Well, in case you missed it, Mozilla recently announced the upcoming release of Firefox 3.5, saying they were scrapping the 3.1 version on account of – you guessed it – the number of bugs in this app. Here’s an excerpt from the story on TG Daily:

…Beta 4 is due for release on April 14, but that date is more of a guideline than a deadline given the number of remaining bugs — nearly 100 were found so far, at least three of which are still “Priority 1 Blockers” (serious bugs).

Fixing is “On Track” for the largest bugs, according to Mozilla and is focused on upgrade compatibility, performance and better controls for video.

Beta 3 is mostly a bug-fix release. However, according to the Mozilla Developer Center pages, Firefox 3.1 will be renamed to Firefox 3.5 after the Beta 3 release. The reason: Pure marketing on one side, and the substantial number of improvements as well as the large number of new features such as private browsing and the Tracemonkey JavaScript engine, on the other.

uTesters gave Firefox 3.1 beta great reviews after the last Bug Battle, and so I suspect 3.5 will have even more enthusiasts this time around. Can’t wait for the launch to think about testing the new version? Then be sure to check out Mozilla‘s QA page.

If you or someone you know is struggling to develop iPhone apps, this recent item from TechCrunch may provide relief:

At the recent Future of Web Apps conference in Miami, Y-Combinator-backed 280 North announced Atlas, a drag-and-drop visual editor for building desktop web applications with Cappuccino, 280 North’s Javascript-based framework…Atlas can wrap up iPhone Web Apps like native applications, granting them access to a significant portion of the iPhone API and allowing them to be sold through the App Store.

This lowers the barrier of entry for iPhone development substantially, allowing those with Javascript knowledge to create fully functional applications on the platform without requiring them to learn a whole new language. The same limitations that apply to Javascript apply here, presumably – in other words, don’t expect to be throwing down ultra-rich OpenGL-based 3D games, but mid-range apps (such as Twitter clients, RSS readers, etc.) should be completely doable.

I’ve gathered the significance of this from a developer’s perspective, but I’m curious as to what the testing community thinks, and three questions immediately come to mind:

1. Will this complicate the QA process for these apps, make it easier, or have no affect at all?

2. Besides Twitter clients and RSS readers, what other types of mid-range apps should I expect to see?

3. Will the lower barrier to entry result in lower quality apps, or just the opposite?

Leave a comment and let us know what you think.

Our Bug Battle of the Social Networks is now open and running.  Starting on Wed, Feb 11th, uTesters can test the quality, features and usability of MySpace, LinkedIn, and Facebook.  The top testers will receive prizes ranging from $200 to $1,000 for showing us the best bugs and feedback — more than $3,500 in total prize money!

Visit our Bug Battle homepage for more details.  Also, make sure you read our Tips to Win the Bug Battle.

So if you’re a uTester, get started today.  And if you’re not yet a uTester, join our community and take part in the fun.  Good luck!

In Q4, we held our first-ever Bug Battle — the “Battle of the Browsers”.  This QA contest yielded 672 bugs within IE, Firefox, and Chrome.  1,300+ testers participated and uTest awarded nearly $3,000 in prize money.  Now it’s time for our next Bug Battle challenge.

After a great deal of discussion with bloggers, analysts and members of our testing community, we’ve selected a fun and high-profile “battleground”:  Social networking sites.

Join other uTesters from around the world in testing the application quality and usability of the top three social networks:  Facebook, LinkedIn and MySpace.

The contest begins Wednesday, Feb 11th and runs through Tuesday, Feb 17th.  uTest will once again be awarding $3,000 in prize money for:

• Top overall tester (based on quantity and quality of bugs and feedback)
• Top novice tester (same criteria)
• Top individual bugs in each of the sites (highest profile, most interesting, most severe)
• Best feedback (about the feature set, functionality and usability)

See complete details for the uTest Social Networking Bug Battle.  Get ready to hunt some bugs, make some serious money, and be recognized as one of the top testers in the community!

Have questions or ideas for future Bug Battles?  Drop us a comment.