Software Testing Blog

Hello All,

This week, I wanted to take some time to write about the benefits of community QA testing and compare it with other, traditional models of QA. Three traditional methods of QA testing are: in-house, outsourced (typically professional firms), or Beta testing.

In-House:

In house testing does have some very positive benefits which include: rigorous testing, fully managed teams, and QA personnel being fully knowledgeable about the product. Unfortunately, the team is generally excessively under-or-over utilized given the exceptionally cyclical nature of the QA environment. In other words, when a release comes out, the team is expected to work at maximum capacity and they are always under extreme pressure to either get the product back to engineering, or to release it to market. Conversely, in-between released, the team is highly under-utilized. Clearly, this kind of cyclical work flow incurs a high amount of cost, and relatively low efficiency for the money being put into it.

Outsourced:

Outsourcing QA to professional QA firms does solve the “peak workflow” issue to some degree. Unfortunately, they usually incur an even higher annual cost than maintaining one’s own QA department. They do tend to be highly skilled, yet generally they lack familiarity with one’s product, are very expensive, and will not have as much incentive to find bugs as the community testing model.

Beta Testing:

Beta Testing is by far the cheapest of the above mentioned techniques. Companies can typically receive extremely useful feedback since it is being tested by those who will be using it. Unfortunately, therein lies the problem in that beta testing requires using one’s own consumer base to test its product. If it is exceptionally buggy, many users might become upset with the product quickly, subsequently generating bad PR for the firm. The testers are also not controlled or managed very effectively. There are a number of ways to organize a beta testing environment, but unfortunately none of them are as well organized as either having one’s own QA department or outsourcing it.

uTest Community Testing:

The reason why we are so excited about our community testing model (as one uTest supporter recently called it – a “team testing model”), is that it seems to offer the best solution by far to all of the aforementioned problems. It is extremely easy to manage one’s cyclical work flow because the company can scale up to as much labor as they need, when they need it, subsequently reducing their time to market by a large factor. It is much less expensive than an in-house or outsourced QA team, but still offers the ability to be just as tightly managed. It offers the inexpensive means of beta testing without using one’s consumers as test subjects, and the ability to not run over budget.

We appreciate all of your continued enthusiasm and support throughout our release and we look forward as always to hearing your input. If you know of any companies that might be able to benefit from our service (and therefore provide more work opportunities for our testers), please let us know at sales@utest.com. Thank you very much and we look forward to writing more soon.

Best,

Jonathan and the uTest Team.