Software Testing Blog

As the Community Ambassador here at uTest, I spend the bulk of my time conversing with testers from our office here in Southborough, Massachusetts. So you can imagine how excited I was when I got the chance to host an off-site tester meetup – in Tokyo of all places!

I was already planning to vacation in the area, and so I decided to “advertise” the event in the uTest Forums. Since I posted the meetup on relatively short notice (10 days), and seeing how we have only launched a handful of releases in Japan (so far), I was somewhat unsure of the response I would receive. Well, after five days, I had been contacted by five our testers, and so we decided to meet at the world’s busiest intersection: Shibuya’s famous Hachiko area.

So how, you ask, were our fellow uTesters able to find each other in such a hectic setting? Easy – the uTest t-shirt. A few minutes before the meeting on a Saturday afternoon, our first Japanese uTester – an Austrian gentleman who has lived in Japan for the past 15 years – flagged me down when he caught glimpse of said t-shirt. Ten minutes later, the other testers showed up and we took our meeting across the street to the Excelsior Café – a nearby coffee shop that is a favorite among the locals.

Read more…

Busy couple of days at the AlwaysOn event (#aoss09) in Palo Alto, CA…

As previously mentioned, we were named to the AlwaysOn Global 250 list of emerging private companies.  Well, at last night’s ceremony, they officially recognized the list, and we were named the overall winner in the “SaaS and Enterprise” category.

This morning, Doron presented the uTest story during the CEO showcase.  The room was packed with startups, entrepreneurs, VCs and media.  Afterward, we conducted demos and discussed, among other things, crowdsourcing, software testing, mobile apps and overall startup life.

This afternoon, we had a series of great discussions with bloggers and journalists, including:

• Steve Bell (@sbell22) from Startup Trek (video interview & article available here)
• Mark Boslet from TechPulse 360 (@techpulse360)
• New York Stock Exchange (NYSE)
  
• Svetlana Toun-Sambell, EDU Research Group

It’s always cool to see what other startups are up to.  It’s even cooler that so many of them approach us, and already know our name and story! If you’re in the Palo Alto area, and want to meet up, please shoot me a note.

More big news from uTest nation today:  we were named to the AlwaysOn Global 250 list.  In fact, uTest was selected as the SaaS & Enterprise category winner.  This category had more than 40 other finalists, including proven and prolific innovators such as HubSpot, Jigsaw, Box.net and SlideRocket.

AlwaysOn surveyed hundreds of venture investors, investment bankers, and technology insiders, which yielded more than 1,000 nominees.  That list was pared down to the AO Global 250 winners list through company interviews and rigorous research.  And from that prestigious list, there were just eight category winners.  We’re honored and ecstatic to have been named the category winner on such a coveted list.

The AO Global 250 winners have achieved, “significant market traction and game-changing technology. In addition to being innovators, the AO Global 250 winners all demonstrated leadership among their peers in the following areas:  innovation, market opportunity, commercialization, media buzz, and stakeholder value creation.”  In AO’s own words, the winners shared a common trait:

Read more…

With the summer sun finally paying us a visit in Boston, so too is the mobile testing space heating up out here! Tomorrow (7/16), Doron Reuveni will be speaking at the 2009 Software Industry Conference (SIC) at the Boston Marriott Quincy on how crowdsourcing is helping companies test their mobile apps.

With the new iPhone 3G S, Blackberry Storm 2, and the new Palm Pre, the smartphone apps competition is fiercer than ever! Mobile app developers are battling for media attention, consumer dollars, VC investments and more.

But, status quo testing doesn’t quite work for mobile apps. Multiple testing criteria — such as wireless carriers, handset makers and models, locations, languages, platforms — all must be taken into account. With crowdsourcing, any mobile app company can have world-class QA coverage.

In his session, Doron will examine the growing trend of crowdsourced software testing for mobile apps – highlighting its benefits, challenges and uses as compared with conventional QA options.

If you’re in town and want to meet up, shoot us a note!

And by trophies, we mean nearly $4000 in cash prizes.

Yes, the results of the Battle of the Desktop Twitter Apps are final. The week-long QA competition – in which more than 600 testers  searched for bugs in five of the most popular Twitter apps – was a  success. As expected, we received hundreds (320 to be exact) of interesting GUI, technical and functional bugs, so selecting a handful of winners was obviously no easy task.

In fact, it was so difficult that we decided to expand our winners list. Also, be sure to download our free special report (PDF), detailing the results of the post-battle usability survey.

As you’ll see from the results page, the Battle of the Desktop Twitter Apps saw several repeat winners, along with some fresh, new uTest talent. Here’s a top-line list of the winners:

• Top Tester: Bernard Lelchuck (Israel)
• Top Novice: Tyler Ritchie (USA)
• Top US Tester: Joseph Ours

Tyler and Bernard also finished 1st and 2nd in the “Best Bug” category. Other winners for “Best Bug” included:

• Claire Pelayo (Philippines)
• Bryan Fisk (New Zealand)
• Aymen Chehaider (India)

As far as the apps were concerned, the uTest community ranked Tweet Deck #1 in terms of usability, feature set and overall quality. Coming in second (also in every category) was Seesmic Desktop, followed by Twirl, Tweetr and Twitteroo.

The results have already been featured in a Mashable story yesterday, ase well as a bunch of other news outlets, and of course, hundreds of individual tweets.

Once again, congratulations to all of the Bug Battle winners, and thanks to all those from our community who participated. If you’re aleady looking forward to our next competition – and we know you are – be sure to send us your ideas. We can be reached at marketing@utest.com, and of course, on Twitter.

Think crowdsourcing is a fad?  Think it won’t last?  Then don’t read the latest edition of BusinessWeek, which has a couple of insightful pieces on the topic from Jon Winsor.  While it’s widely acknowledged that the current global economic conditions have enabled B2B crowdsourcing to grow more quickly than they would have otherwise, Winsor believes that this community-driven movement will outlive any economic cycle:

Some might think once the global financial crisis has come to an end these marketplaces will dwindle as people go back to work.  Instead, I believe that they will evolve further, supplying a more efficient and creative way for companies to engage with and harness the crowd for help.

uTest is included in BusinessWeek’s article on crowdsourcing leaders and the related slideshow uses a great quote from our CEO.

You can probably guess what we think about this topic:  open marketplaces and crowdsourcing are the most efficient, frictionless, collaborative way to deliver professional services like software testing!  But we’d love to hear from you.  Is crowdsourcing the natural evolution of outsourcing (as James Whitaker has said) and thus, here to stay?

After web browsers and social media sites, you were probably wondering how we’d top ourselves for the next uTest Bug Battle. Well, after months of debate and deliberation, uTesters will compete to find bugs in six of the top Twitter desktop apps!

Unless you’ve been asleep, in a coma, or camping in the wilderness for the past two years, you’re likely aware that the Twitterverse is expanding at a mind boggling rate (thanks, Ashton and Oprah). And since hundreds of apps have been created around the popular micro-blogging site, we figured they’d make a perfect subject for our now famous Bug Battle. Here are the Twitter apps we’ll be testing, in no particular order:

1.    Tweet Deck 0.25
2.    Seesmic DESKTOP
3.    Twhirl 0.9.2
4.    Tweetr 3.4
5.    Twitterific 3.2 (mac only)
6.    Twitteroo 1.5 (pc only)

The contest kicks off RIGHT NOW (12:01am ET on Thursday) and will run through 11:59pm ET next Wednesday, June 3rd.

During that time, members of our QA community will be searching for the most serious, compelling bugs they can find, including technical, functional and GUI bugs. uTesters can focus their efforts on as many or as few of these as they choose.  We will be awarding more than $3,000 in prize money for:

* Top overall tester (based on quality of bugs and feedback)
* Top novice tester (same criteria)
* Top US-based tester (same)
* Top five individual bugs (severity and complexity)
* Best feedback (post-contest survey about the feature set, functionality and usability)

You can get more info on the Twitter Bug Battle. To find out how you – yes you! – can win the money finding bugs in these Twitter applications, be sure to check out the Bug Battle thread in our testers-only forums.

Have questions or ideas for future Bug Battles? Drop us a comment. Want to keep up with the Bug Battle action while it’s going on?  We’ll be sharing real-time updates throughout the weeklong Twitter contest on (where else?) Twitter.

The word is out!  With the launch of our revamped website, a new “Meet the Testers” application and Tester Forums, last week uTest made significant strides in turning our crowd of 16,000+ QA professionals into an interactive community.

Here’s what the media had to say:

• Jeff Howe, author of best-selling book Crowdsourcing & contributing editor at Wired Magazine, twittered the news to his 2,000 followers calling us out as a Company to Watch.

• Xconomy’s Wade Roush highlights uTest putting a human face on software testing with the new site’s collaboration tools in his story, uTest Site Revamped for Collaboration.

• VCcafe’s Eze Vidra focuses on the “Meet the Testers” app & its drill-down capabilities “into a 16K strong testing community” in uTest Puts a Face on its Crowdsourced QA Community.

Other notable posts at  StickyMinds.com, the Cloud Architect, and MSNBC.com.

Let us know what you think about the new site!

This evening, Doron Reuveni will be joining a new meetup here in Boston called the Ultra Light Startups. The meetup is a resource for entrepreneurs to share best practices for launching tech startups. The topic is near and dear to our hearts: Crowdsourcing. If you’re in the neighborhood, come join us at Boston University at 595 Comm. Ave. Doors open at 6pm! Questions the panelists (Local Motors, Acquia, GeniusRocket and uTest) will tackle:

• What is crowdsourcing?
• What are the benefits?
• Why is it a disruptive model?
• What are the most effective ways to build communities?

Doron will also be flying down to Orlando to present at STAREAST 2009 on Thursday morning. His presentation will introduce this new era of community-based software testing and delve into how companies can launch higher quality apps while staying within budget through crowdsourcing. Challenges like shorter release cycles, increased customer expectations, smaller budgets and fewer testing resources are forcing us to rethink our QA methods.

Doron will discuss how crowdsourced testing helps to meet these challenges head on. Here’s your chance to take the kids to Disney and be a part of the largest and most advanced testing forum to keep you up on the latest trends, technologies, and strategies in the industry today. Shoot us a note if you’ll be down there!

As we mentioned previously in this space, uTest was recently chosen to present at the Under the Radar event in the heart of The Valley.  With only 24 start-ups being handpicked from among hundreds to present at UTR, just being selected is an honor.  Their track record for finding future stars before they pop is beyond impressive.

So last Friday afternoon, in front of a crowd of 200+ start-up execs, VCs and members of the media, Doron took the stage to tell the uTest story — and to do it in six minutes or less.  At an event that was focused almost entirely on cloud computing (infrastructure, management, apps, etc.), we knew our vision of crowdsourced software testing would be the exception and we wondered how it would be received amidst all the cloud talk.

And while I’m unabashedly biased, my initial reaction was that our story was popular within the room.  Apparently I was right because, later that day, we learned that uTest had won the judges’ choice award in the “Applications” category.  A high honor indeed, as our judges were such notables as John Foley from InformationWeek, Matthew Glotzbach from Google and Robin Vasan from Mayfield Fund.

The entire experience was positive — some great meetings with prospective customers and partners, good meetings with members of the media, and another feather in our cap as we rewrite the rules of what’s possible in the world of software testing.

Check out the six minute presentation for yourself and drop us a comment with your thoughts.