QAZone, the largest online community dedicated to fostering communication and collaboration among QA & IT professionals has released an interview with uTest founders and management team. You can view the QAZone post here:
We are working with QAZone on future collaboration between the uTest and QAZone community and sites. New announcements are soon to follow. Marina Gil-Santamaria from QAZone described it best: “QAZone is the community that can support you while you learn about testing/QA methodologies and best practices, and uTest is the community where you can apply your knowledge hands-on and get paid for bugs that you find!”
For your ease of reading the interview in its entirety is posted below:
QAZone Interview with Doron Reuveni, Roy Solomon &
Marc Weinstein
-March 2008
Q1- Doron, Marc, Roy, you started a new company, uTest, built on the philosophy of building a testing community and offering QA as a service….Do you want to start by talking about uTest, and giving a high level overview of your company & business model for our QAZone members & visitors?
A1- (Marc) Marina, first and foremost than you for this opportunity to introduce uTest via QAZone. uTest has introduced the first global marketplace of QA software testing coupled with a Pay Per Bug business model. Companies in need of software QA testing are able to access a Global Community of professional software testers - on-demand - for the purpose of quality assuring and testing software releases prior to market launch.
The uTestTM testing platform is all Web-based - accessible anywhere, anytime, from any computer screen by customers and testers. It enables application vendors to load/publish their application, test scripts and testing information to the secure uTestTM Testing Platform. The interface is easy to use and allows companies to self manage QA cycles, projects and processes.
The pay per bug business model introduced by uTest is unique in the QA testing world, but is used very successfully in other industries and in other aspects of technology including software development. We believe this model is highly suitable for QA software testing based on the feedback from our customers and testers during our pilot program.
Q2- Who came up with the idea? Was uTest the answer to some industry trend that you noticed, were you trying to find yourself qualified QA engineers yourself?
A2- (Marc) Marina, great question! Roy would you briefly summarize the uTest story….
(Roy) “Yes, thank you Marc. The uTest idea was first conceived by me (Roy). I managed several large QA teams testing applications geared for consumer markets utilizing both in house and outsource QA resources. Significant testing hours were invested prior to market release only to find many new bugs on a post-release basis by customers. What materialized was the fact that true end-user environments and real-world user-scenarios could never be sufficiently tested prior to market release due to time-to-market and budget constraints.
I tried to solve this problem by running extended beta programs, but as with other beta programs, feedback often remained insufficient, unreliable, lacked real-world behavior, and further delayed product launch. It didn’t take too long to realize that this problem could be effectively and efficiently solved with a diverse global community of professional testers available on-demand”.
Q3- Let’s focus now on the tester’s side of uTest…What are some of the skills/characteristics that you are looking for? Are there any conditions (like lack of experience, lack of automation skills, etc) that will “block” a tester from joining uTest?
A3- (Doron) “As Marc noted earlier, we currently have a global community of professional software testers that is in excess of 5,000 and growing daily! Their experience ranges from Novice to expert QA testers (with 10+ years experience). We offer an open sign-up to participate in uTest projects.
Most customers require a blend of both experience and novice testers. There are often customers and projects which require unique configurations and tester experience, this provides a tremendous opportunity for novice testers to participate and improve their testing skills and experience”.
Q4- How many uTest members do you typically assign to work on the same project? Are they all working on the same area, do you divide an application in sub-areas?
A4- (Doron) “We currently assign 30-50 testers per project to view and test all parts of the application. However, our testing platform will soon be enhanced to send real-time indicators of testing coverage that will alert and direct testers to different areas of the application”.
Q5- You are paying using a per bug model, so how do you handle when more than one tester reports the same issue? Do they have visibility into each other’s bugs?
A5- (Doron) Marc, would you provide some highlights of the uTest Platform…
(Marc) “The uTest platform has a built-in duplication alert mechanism. Prior to posting a known or duplicate bug, the uTest system will alert the tester of potential duplicates and advise the tester not to report them and/or provide additional information. The tester community does have access to all known bugs per release, unless the company specifically decides not to share this information”.
Q5- I know that coming up with a compensation number might be hard as there are multiple factors to consider, but let’s say that we define “active” as dedicating 20 hours/week to uTest. What could be a typical compensation range in this scenario?
A6- (Marc) “The tester bug rate is determined by tester grading, tester profile, type of test release, number of testers assigned to a release, and other testing criteria defined by the company. On average, 20 hours a week would earn a tester between $200-$500 a week. However, more experienced testers can earn significantly more. We believe, and our market research has shown, that the income potential coupled w/flexible work hours and the opportunity to work on multiple and new/exciting products will exponentially grow our tester community. We also facilitate special bonuses and additional incentives to top testers participating in a QA project based on amount of bugs or importance of bugs found”.
Q6- Do you provide documentation & resources around testing methodologies/best practices for your members? Are your uTest testers able to interact with other for guidance, discussions and peer to peer support?
A6- (Marc) Doron, would you more fully address this matter…
(Doron) “Best practices and resources focused on testing methodologies are available via the uTest Blog and FAQ section of the uTest platform. uTest, as well as its testers, are using QAZone as a great resource center for methodologies and best practices around QA and testing. As part of the uTest platform, we also provide a messaging component that enables testers and companies to communicate. We also plan to introduce a “forum system“ that will allow testers to communicate amongst themselves along with the company Testing Manager”.
Q7-I am glad that you mentioned QAZone! J. Now if we focus on the companies that will benefit from signing up for your uTest services…What do you think is the ideal company/application profile?
A7- (Doron) “We truly believe that any company that develops a product targeted for consumers can significantly benefit from the uTest services. We are currently focusing on Web and desktop application providers, but we are also developing components that will support smart phone and PDA-type applications and companies that provide packaged hardware and software products to the consumer (i.e. music, enhanced digital devices, etc.)”.
Q8- In your opinion, what is the major driver for companies that are signing up for your testing services? Do you think that they are looking to reduce headcount on their existing QA teams on the longer term?
A8- (Marc) Marina, again great question…”Companies drawn to uTest recognize the tremendous business value in leveraging an unlimited, diverse, and experienced community of professional software testers on-demand. Unlimited not only by the number and diversity of testers, but by the variety of testing characteristics; Environment, OS, Browsers, Extensions, Anti-virus, Firewall, etc. uTest is a “true” complement to a company’s existing QA efforts who turn to uTest for an out-of the-box QA experience, access to testing environments not available in-house, testing across the end user food-chain - from Novice to Expert - during peak periods, and to better manage time-to-market expectations”.
Q9- Is there any type of project that is better suited for uTest? (E.g. functional testing, Beta testing, Performance testing, new applications, Beta cycles, etc)
A9- (Roy) “Initially we are focusing on functional, beta, acceptance, and exploratory testing. We fully anticipate introducing load testing and usability testing as well”.
Q10- Roy, Doron and Marc, thank you very much for your time! It was truly a pleasure speaking with you today. I would like to wrap-up this interview with a last question…. In your opinion, what is the future of Testing/QA as a service?
A10- (Doron) “We believe the trend that we see in QA software testing will be similar to what we see in other technology segments. We will see more and more SaaS-type solutions being offered on an on demand basis. We also believe that the pay per performance model will provide tremendous business value to both companies and testers”.