Software Testing Blog

Whether you’re a crowdsourcing critic or devotee, it’s worth hearing all the angles from the experts and from those who have built crowdsourcing business models. Check out the Top 20 Crowdsourcing Tweeps (experts and companies) to follow on Twitter below (in no particular order). Also, here are a few recent articles to hopefully spark even more debate — The Huffington Post’s Does Crowdsourcing Threaten Your Job (or Offer New Opportunity)?, Entrepreneur Magazine’s Crowdsourcing: Opportunity or Time Suck?, and Network World’s Could designer bras be a natural fit for crowdsourcing?.

1. Jeff Howe, @crowdsourcing (coined term crowdsourcing, quiet this summer)
2. LiveOps, @liveops
3. Neil Robertson, @nielr1 (co: @Trada)
4. John Winsor, @jtwinsor (co: @VictorsnSpoils)
5. Ross Kimbarovsky, @rosskimbarovsky (co: @crowdspring)
6. Dwayne Spradlin, @InnoCentiveCEO (co: @innocentive)
7. Tim Thomas, @imstarboard (co: @localmotors)
8. Community Roundtable, @jimstorer & @rhappe (co: @TheCR)
9. 99 Designs, @99designs
10. CrowdFlower, @crowdflower
11. uTest, @uTest (shameless self plug )
12. Chokha, @chokha
13. Mike Martoccia, @mmartoccia
14. Top Coder, @TCJim (co: @_TopCoder_)
15. Threadless, @threadless
16. Chaordix, @chaordix
17. Peter Lamotte, @peterlamotte
18. Genius Rocket, @GeniusRocket
19. SmartSheet, @crowdwork & @mcolacurcio
20. Tongal, @tongal

Update! Apologies for those I may have missed. Here are more recommendations straight from the community: @crowdsourcecap, @ArticleOne, @crowdsourcerisk, @fergusdyersmith, @OpenRunway, @crowdsourcery…

More can be found on our actual Crowdsourcing Twitter list. So, who do you follow on Twitter to find the best crowdsourcing advice and breaking news? Did I miss anyone? Please let us know in the comments and I’ll add them to the list.

Not to beat a dead horse or anything, but I wanted to briefly revisit Apple’s  “Antenna-Gate” fiasco to drive home a very important lesson for companies of all shapes and sizes: Rely too heavily on “lab-testing” and you are virtually guaranteed to get burned.

We recently learned about Apple’s “Top Secret” design and testing lab thanks to MG Seigler of TechCrunch, who was given access to the state-of-the-art facilities just days before he mysteriously disappeared (kidding).

For some, the futuristic lab has conjured up images from the movie Star Gate, although I think it looks more like the Senate floor from Star Wars (episodes I through III). Here’s Seigler with a more technical description, as well as some insight into how Apple actually uses it:

Inside Apple’s headquarters in Cupertino, CA, there are a collection of rooms that house 17 giant anechoic chambers. Basically, they’re rooms where no waves (sound or electromagnetic) can reflect off of anything, so there is absolutely no interference when it comes to wireless testing. Apple places their devices from iPhones to iPads in these chambers to ensure the performance is up to their standards.

So how do they test it? There are four stages. The first is a passive test to study the form factor of the device they want to create. The second stage is what Caballero calls the “junk in the trunk” stage. Apple puts the wireless components inside of the form factor and puts them in these chambers. The third part involves studying the device in one of these chambers but with human or dummy subjects. And the fourth part is a field test, done in vans that drive around various cities monitoring the device’s signal the entire time (both with real people and with dummies).

So where did Apple go wrong? And what can this controversy teach us about the difference between in-the-lab-testing vs. in-the-wild testing? Below the jump are four critical lessons that companies ignore at their own peril:

Read more…

Last week, we posted a pretty powerful piece about the loyalty of crowds for all those crowdsourcing veterans out there — both skeptics and champions alike — to start demystifying the “human nature” of online communities. We received some really great feedback (thank you!), and realized that many people still haven’t heard the basics of the crowdsourcing model. For those of you who just need the quick nuts and bolts, here’s a pretty good interview on CBS NEWS that may be useful: What Is Crowdsourcing?

Watch CBS News Videos Online

Depending upon who you ask, crowdsourcing is either evil, revolutionary, or a next gen of internships.

But one thing that ALL crowdsourcing companies like to preach is how loyal and trustworthy and professional their community is. I know because I’ve read it in 100 different sites. Hell, I’ve written it a 100 different times here at uTest. So why do crowdsourcing companies insist upon telling the world how loyal and earnest their community is?  Maybe it’s to assuage the fears of prospective customers about entrusting their logo design, app development, content production or marketing to a community of strangers. Maybe it’s because if marketers say it enough times, we hope it’ll come true.

The more pessimistic view is that people — cloaked in the anonymity of the web — often act in greedy, selfish, mean-spirited ways (this perspective didn’t make it into the crowdsourcing brochure, by the way). Such dark behavior is well-documented and takes the form of flame wars on message boards, bullying via social media and online fraud.

So which is it — are people good-natured and honest?  Or are they money-hungry malcontents who will do anything to get ahead, as long as they don’t get caught?  Obviously it depends on the people, but I learned the truth about our community this week — and it was a lesson we learned the hard way.

A little background:  At uTest, we pay our testers twice per month via PayPal or Payoneer. And at this point in our growth, each pay cycle involves a non-trivial amount of cash — pretty deep into the five-figure range.  Now, it’s not easy or flattering to admit this, but in our most recent pay cycle, we experienced a glitch that caused us to pay our testers twice. That’s right folks, it was double payday here at uTest!

Read more…

Well, today he’s sticking close to home in Boston. Tomorrow he’ll land in London… and before the week is out, he’ll hit Tel Aviv.

Doron starts Wednesday morning off (after his usual 10-mile run, of course!) in London with some tea and networking with friend and colleague, James Whittaker and UK partner, TCL.

Then he’s off to QCon London, an excellent conference for the enterprise software community. On Friday, 3/12 @ 2pm, he’ll be presenting at QCon re: The Mobile App Quality Challenge & How Crowdsourcing Can Help.

Doron is one of five software testing leaders chosen to present in the “How Do You Test That?” track. This track explores unique solutions created to address situations in which automated testing does not suffice.

And on the last leg of his marathon journey, Doron will present at Garage Geeks in Israel on Monday, 3/15 @ 8pm. There, Doron will be taking a deep dive into the topic of Crowdsourcing, and how smart recruiting, training and incentives can turn an unstructured, loosely assembled mob into a unified, professional community.

So, where in the world is Doron this week?  Catch him if you can!

You may have noticed that many top reporters and bloggers included “crowdsourcing” as a top trend of 2009. In fact, The New York Times named it 2009’s verb of the year!

But what will separate crowdsourcing’s winners and losers in 2010?  We believe that crowdsourcing success lies in a company’s ability to engage its community in unique and meaningful ways. These are the crowdsourcing companies that are changing the way work is done today — transforming professional services like software testing forever. But how does a company go beyond building a “mob” or a “crowd” and develop a community?

VOTE HERE to learn more about how crowdsourcing is changing the way work is done.

We’d love the opportunity to share our hard-earned keys to building a successful crowdsourcing business at this year’s Enterprise 2.0 Conference in Boston. But first, we need your help! (Very quick registration required.)

You can see a snapshot of uTest’s proposed presentation on the voting site. Voting opens today and closes on January 20th. As always, thanks for your continued support. All of our success stems from our amazing community!

Quick Update: If you’re posting online, #e2conf is the official tag! To track votes, click here.

Breaking news! This week, we officially announced the results of our Q4 Bug Battle – Battle of the E-Tailers – along with a few prominent media outlets, including USA Today, Fast Company, Mashable! and eWeek.

Curious about the results? Check out the articles below:

• Study: Amazon Most User Friendly by Byron Acohido (@lastwatchdog)
  

• Amazon Trumps Target, Walmart in Bug Battle by Kate Rockwood (@katerockin)

• Holiday Shopping: Amazon FTW! [STUDY] by Christina Warren (@film_girl)

• Amazon Is Least-Buggy Retail Site, Says Report by Nicholas Kolakowski (@eWEEKNews)

So, the battle begs the question: Which e-tailer do you trust the most?

Up there — it’s a bird… it’s a plane… Nope, it’s just the latest experiment to measure the ability of crowdsourcing to organize, mobilize, collaborate and compete.

DARPA (who along with Al Gore, invented the Internet), announced that it will place a large, red balloon at 10 different locations around the U.S.   The DARPA Network Challenge calls on groups to locate each of 10 red weather balloons scattered around the country — with $40,000 in prize money being awarded to the first team to accurately identify them all.

The purpose of this contest is to discover how social networking, crowdsourcing and other technologies can help accomplish a large-scale, time-critical task.  Already teams are forming, money is being spent, and the social media universe is abuzz about the competition, which launches on Saturday, December 5th.

Want to know more?  Check out what CNN, CNET, the Wall Street Journal and Scientific American have to say about the competition.

So if you see a big red orb tomorrow morning, call me!  And if nothing else, at least a Google search for the term “balloon” will return something other than another article on the plight of Balloon Boy!

uTest was on fire at Under the Radar Mobility this year. I think Under the Radar said it best!

For anyone looking to deploy an app across multiple mobile platforms and a gazillion different handsets, one massive problem awaits them: QA. uTest solves this problem with an army of testers across the world. Crowdsourced QA… Problem solved. (Click here to see Doron’s presentation.)

And that’s not all! Doron was able to connect with multiple partners, prospects and top media outlets, including Mashable’s Ben Parr (@benparr), editor in chief at IntoMobile.com Will Park (@willpark), ReadWriteWeb’s Dana Oshiro (@suzyperplexus), as well as participated in a couple great video interviews with bnetTV’s Michelle Sklar (@bnettv) and GoMo News’ Cian O’Sullivan (@gomonews) which are posted below!

Take a peek at the video interviews below to learn more:

Doron Reuveni-CEO of uTest speaks with bnetTV.com at the Under the Radar event.

Read more…

As promised, Google has made the slides and video presentations from GTAC 2009 (Google Test Automation Conference) available on the GTAC website and on YouTube. This year’s GTAC was a huge success! The theme was “Testing for the Web,” and now anyone can watch these leading thinkers discuss test automation strategies, tools, and the challenges desktop and mobile environments present when creating web apps.

Doron was among a select group of speakers chosen to present at GTAC, including Microsoft, smartFOCUS Digital, Sauce Labs and of course Google, where he examined the complimentary role a community of professional testers plays in mobile testing.

Check out Doron’s presentation below! All other presentations can now be seen on YouTube.