To Crowdsourcing Friends, Foes & Fanatics: Just How Loyal Is Your Community?

Depending upon who you ask, crowdsourcing is either evil, revolutionary, or a next gen of internships.

But one thing that ALL crowdsourcing companies like to preach is how loyal and trustworthy and professional their community is. I know because I’ve read it in 100 different sites. Hell, I’ve written it a 100 different times here at uTest. So why do crowdsourcing companies insist upon telling the world how loyal and earnest their community is?  Maybe it’s to assuage the fears of prospective customers about entrusting their logo design, app development, content production or marketing to a community of strangers. Maybe it’s because if marketers say it enough times, we hope it’ll come true.

The more pessimistic view is that people — cloaked in the anonymity of the web — often act in greedy, selfish, mean-spirited ways (this perspective didn’t make it into the crowdsourcing brochure, by the way). Such dark behavior is well-documented and takes the form of flame wars on message boards, bullying via social media and online fraud.

So which is it — are people good-natured and honest?  Or are they money-hungry malcontents who will do anything to get ahead, as long as they don’t get caught?  Obviously it depends on the people, but I learned the truth about our community this week — and it was a lesson we learned the hard way.

A little background:  At uTest, we pay our testers twice per month via PayPal or Payoneer. And at this point in our growth, each pay cycle involves a non-trivial amount of cash — pretty deep into the five-figure range.  Now, it’s not easy or flattering to admit this, but in our most recent pay cycle, we experienced a glitch that caused us to pay our testers twice. That’s right folks, it was double payday here at uTest!

Continue Reading

I Love The Smell Of Bug Battles In The Morning…

No bugs were harmed in the planning of this Bug Battle… but they will be — starting Friday at noon ET.  That’s right uTesters — it’s time for our 2nd quarter Bug Battle competition!

And this time, we’ll be checking out the top location-based check-in services: Foursquare vs. Gowalla vs. Brightkite.  We’ll be letting our community put these innovators to the test (their public-facing websites, their check-in services, their mobile apps and their integrations with Facebook and Twitter).

And after the testing phase is complete on Monday, May 24th, we’ll be sending a survey to all participating testers to compare the usability and feature set of these three leaders.

What’s in it for testers?  Well, bragging rights, sure.  But we’ll also be doling out nearly $4,000 in prize money to the testers who report the most severe/interesting bugs and provide the most insightful survey feedback.

So it’s time to do what you do best, uTesters!  Log into your uTest account and scour these apps for quality defects and report them in a clear, concise way.  And if you do it better than your peers, you could be named the Q2 Bug Battle winner and earn some big prize money for your time.

uTest Up For 3 Stevie Awards, Including “Most Innovative Company”

Last year, we were psyched to take home the “Best New Company of the Year” for 2009 at the American Business Awards (a.k.a. The Stevies). This year, we’re even more fired up to let you know that the members of the Stevie Awards’ Board have named uTest a finalist in three top categories:

1. Most Innovative Company Of The Year
2. Best Blog in the Computer Services category (for this blog) and
3. B2B Communications Campaign Of The Year
(for the Bug Battle!)

Although we’ve made the shortlist, we definitely have our work cut out for us to bring in these big wins. Take the Comm Campaign category for example — we’re up against huge software leaders, including EMC and Novell. See the complete list of finalists here!

The American Business Awards are the U.S.’s premier business awards program. More than 2,700 entries in virtually every industry were submitted for consideration this year. The winners will be announced on Monday, June 21 at the Marriott Marquis Hotel in Times Square/NYC. More than 600 execs from around the globe are expected to attend.

Let us know if you’ll be in NYC that week. We’d love to meet up and fill you in on the latest!

Q2 Bug Battle: Foursquare vs. Gowalla vs. Brightkite

Where are you going, where have you been? Yes it’s the title of a creepy short story, but it also happens to characterize the  fastest-growing segment of consumer applications.

Location-based apps like Foursquare, Gowalla and Brightkite – apps that let you check in, find hot spots and get rewarded with prizes & accolades – are attracting thousands of new users every day. Just how popular have they become? They’re sooo popular, even uTest CEO and co-founder Doron Reuveni has attained ‘Mayor’ status (see if you can guess where).

Why am I telling you all this? Because these  geo-based apps are the subject of our latest uTest Bug Battle. We figured that since Foursquare, Gowalla and Brightkite are all global in scope, that they would provide our global tester community with a challenge unlike any of our previous bug-hunting competitions.

The Bug Battle gets started this Friday, May 14 at Noon (ET). Testers will be given ten days to search these applications (both web and mobile) for the most compelling bugs, and to report them through our online platform. We’ll be dishing out nearly $4,000 in prize money for Top Tester, Best Bug, Best Survey Feedback, Best Mobile Bug and a bunch of other categories.

More details on this Bug Battle – as well the rules, prizes and deadlines –can be found in this uTest Forums thread. Of course, we’ll be providing frequent updates via our blog, forums and newsletters.

Special note: This competition is available ONLY to members of the uTest community. Not a member yet? Find out more about the benefits of joining uTest.

Top 20 Software Testing Tweeps

According to Twitter co-founder Biz Stone, Twitter now has 105,779,710 registered users—and is adding 300,000 new users a day. Attempting to weed through all of the fluff can be daunting! So, if you’re interested in jumping into the Twittersphere or are just looking to follow the leading journalists and thinkers in software testing today, check out our “Top 20 Software Testing Tweeps” list below (in no particular order)!

  1. James Bach – @jamesmarcusbach
  2. Michael Bolton – @michaelbolton
  3. Testing At The Edge Of Chaos (Matt Heusser) — @mheusser
  4. Tester Tested! (Pradeep Soundararajan) – @testertested
  5. StickyMinds.com (Better Software Mag) — @StickyMinds
  6. SearchSoftwareQuality.com (Yvette Francino) — @yvettef or @SoftwareTestTT
  7. Google Testing Blog (Copeland/Whittaker) – @copelandpatrick or @googletesting
  8. Testy Redhead (Lanette  Creamer) – @lanettecream
  9. Test Obsessed (Elizabeth Hendrickson) — @testobsessed
  10. SD Times — @sdtimes
  11. Jon Bach – @jbtestpilot
  12. Software Test & Performance Mag –- @STPCollab
  13. Software Testing Club (Rosie Sherry) — @rosiesherry or @testingclub
  14. Lisa Crispin — @lisacrispin
  15. Fred Beringer — @fredberinger
  16. uTest (shameless plug! ;-)) — @uTest
  17. Weekend Testing (Santhosh/Parimala/Ajay) – @weekendtesting or
  18. Santhosh Tuppad — @santhoshst
  19. Ajay Balamurugadas — @ajay184f
  20. Parimala Shankariah — @curioustester

Update! Thanks for everyone’s recommendations. Here are a few we missed: @sbarber, @QualityFrog, @dailytestingtip, @sdelesie, @Rob_Lambert, @chris_mcmahon, @hexawise, @marlenac, @shrinik, @sbharath1012, @sellib, @TestingNews.

Please feel free to add any active Tweeps you think we may have missed in the comments! We welcome your recommendations.

SaaS & Cloud Computing Pioneers Showcase Their Innovations

The OnDemand event in Palo Alto provided the stage where the top Internet companies disrupting the enterprise converged with the incumbent pioneers in cloud computing and SaaS. Kicking off the two-day event with keynote Marc Benioff, Salesforce.com CEO who ignited the on-demand revolution, sure set the bar high!

These days were packed with investor roundtables, CEO showcases, and panel discussions on the trends and opportunities in SaaS and cloud computing services.

Lucky for those of us who couldn’t make the show, all the presentations (including Marc’s!) were recorded and can be viewed on the video archives site. Check out uTest’s below! For a list of a few other great sessions, continue reading after the bump.

Continue Reading

uTest Makes BBJ’s List of Best Places to Work In 2010

Boston Business Journal just announced their list of Best Places To Work. And with nearly 450 nominees and only 20 companies making the cut in the Small Company category (20-100 employees), it’s a tough list to get on. But we’ve never shied away from a tough competition, so we threw our hat in the ring. And lo and behold, we’re proud to report that uTest has joined this prestigious list of  the best places to work in Massachusetts!

We’re stoked to be on a list alongside global heavyweights like Google, Microsoft, Ritz-Carlton, Comcast, as well as local startup legends like Carbonite and HubSpot. So, how did we do it? Well, from our open bullpen layout to our always-stocked kitchen to tweeting from the slopes,  to a company outing that consists of climbing Mount Monadnock, uTest is not your typical company… even by startup standards.

Continue Reading

uTest & SOASTA Join Forces To Share The Load (Testing)

It’s been a busy few weeks around the hallowed halls of uTest (ok, they’re really more of a beige).  Somewhere between delivering the keynote at the QUEST conference, interviewing testing guru Scott Barber, and launching v3.0 of our platform, we found time to sign a partnership with SOASTA.

For those who don’t know them, SOASTA leverages the cloud to offer on-demand load & performance testing, which enables companies to test the real-world performance of their web apps.  In fact, the Wall Street Journal recently named them one of the top 50 VC-backed companies in all the land.  So what’s a crowd company doing partnering with a cloud company?  A lot, actually.

For starters, this partnership expands upon uTest’s recently announced load testing services.  As our customers and their apps become more sophisticated, they’re turning to uTest to provide them with real-world load testing.  And this partnership with SOASTA enables us to offer a unique package of on-demand load testing and advanced analytics to help our customers identify bottlenecks and points of failure between their apps and their users.

Continue Reading

Version 3.0 – A Better, Faster, More Powerful uTest

Check out uTest 3.0

Click the thumbnails below to see a larger screenshot.

Sign In

New Sign-In Page

Customer Welcome Screen

Test Cycle List

After months of hard work, we’re excited to announce version 3.0 of our testing platform.  This is much more than a simple refresh or a minor upgrade — this is a full-blown rewrite of our testing platform, from the UI design, some cool new features, back through the code and a new, open architecture.  And it’s all based upon feedback from customers, testers and all we’ve learned from 1,000+ test cycles in the past 18 months.

Our goal was to produce a faster, more usable, more powerful experience for uTest customers and testers alike.

So what’s new?

  • User Interface: Our new UI was designed to provide a simpler, richer, more interactive experience with more intuitive navigation from bug-to-bug or test-cycle-to-test-cycle
  • Improved Infrastructure: Our new architecture enables better scalability and, more importantly to you, is designed to provide improved performance (with faster page load times) around the world
  • API and Integration: Our new and open APIs will enable us to rapidly expand our offering and better integrate with our customers, partners and third-party app developers

What’s in it for you?  If you’re a customer, keep reading.  To check out what’s new for testers, you can skip down to the tester features.

Customer Features
Once our customers get past our sleek new UI, we hope they notice the great new features and changes we’ve included.  Two features are really important.

First, we’ve built a new test cycle wizard.  Creating a clear, concise test cycle is the most important thing a customer can do to ensure success, but there are a lot of details required to make a test cycle perfect.  Our new wizard makes this process easy, making sure each part of the test cycle is well-documented.

Second, we’ve greatly enhanced our tester rating system.  Our platform will evaluate the past performance of each tester based upon activity levels (# of test cycles participated in, # of bugs reported, # of test cases completed) and the quality of that activity (bug approval %) and, once we have enough data points, we’ll assign them a rating.  Testers who are in the top 20% of all rated testers will receive a gold, silver, or bronze rating.

Some of the other new features we’ve added in v3.0:

Sign In

Advanced Coverage Report

  • Resizable Columns – On any table or list, all columns can be re-sized and re-arranged by customers using a simple drag-and-drop.
  • Tester Messenger - Enhanced tester messaging tools help you communicate with testers to get what you need from each bug report
  • Testing Coverage Reports – Be confident that you’ve covered every corner of your testing matrix with our new coverage reports. You’ll know what’s been tested and what still needs to be tested in a single informative report
  • Easier Filtering – All test cycles and bugs can be easily filtered to find exactly the information you need
  • Smart CSV Exporting – Export a CSV file with just the information you need about your bugs or your test cycles
  • Multiple Users – Add multiple members of your testing organization easily with our new account management tools

Tester Features
Besides the new UI, we’ve added a lot of cool new features for our uTester community.  However two features are really huge for testers.  The first is our new test cycle wizard that helps testers review test cycles more easily and completely, as well as telling uTest whether or not they intend to participate in that test cycle.  Now when a tester checks out a test cycle, they will be able to indicate whether or not they want to join.

Our second big feature is a new & improved tester ratings system.  In the past, testers were rated with a star system based upon subjective customer feedback.  Starting today, testers who are active will earn a rating and a “Rated” badge.  This shows that these testers are actively participating in test cycles contributing their expertise to the uTest marketplace.  Additionally, those testers who are in the top 20% of our rated testers will receive a gold, silver, or bronze badge.  The remaining testers will receive a “Rated” badge.  Testers who are not active or for whom there is not enough data will not receive a badge.  New testers will receive a “New” badge.

A tester’s rating will depend on many factors including total activity and recent activity (# of cycles participated in, # of bugs reported, # of test cases completed), as well as the quality of that participation (bug approval %, test case approval %, and accuracy in specifying bug types and severity).

Other great new features include:

Sign In

Tester Messenger

  • Resizable Columns – For any table or list, all columns can be re-sized and re-arranged using a simple drag-and-drop.  For example, a tester can move the test cycle name to be in the first column with a width that displays the entire test cycle name
  • Tester Messenger – Enhanced communication tools between testers, uTest project management and customers
  • Easier Filtering – All test cycles and bugs are easily filtered.  For example, testers may filter bugs by “My Bugs Only,” status, severity, and by test cycle; even for closed test cycles
  • Clearer Test Cycles – All test cycles are formatted in a more digestible way to help testers more easily and completely review each test cycle
  • Improved Attachments – Bug attachment size limit has been raised from 5MB to 10MB (there is a limit of three attachments per bug).  Also, more file types are permitted for upload.
  • Smarter Emails - Cleaner, more informative notification emails from the uTest platform

Wrap-up
Our engineers and product managers have worked countless hours putting together version 3.0, but we also want to thank our customers and testers for their countless ideas and insights.  As we continue to improve and refine our platform, we are always open to thoughts and ideas about how to make our product better.

Have a great idea for our future product releases?  Testers should join our testing forums and check out our Platform Feedback section. Customers can contact their project manager or drop us a line.