Software Testing Blog

Last week, we posted a pretty powerful piece about the loyalty of crowds for all those crowdsourcing veterans out there — both skeptics and champions alike — to start demystifying the “human nature” of online communities. We received some really great feedback (thank you!), and realized that many people still haven’t heard the basics of the crowdsourcing model. For those of you who just need the quick nuts and bolts, here’s a pretty good interview on CBS NEWS that may be useful: What Is Crowdsourcing?

Watch CBS News Videos Online

Depending upon who you ask, crowdsourcing is either evil, revolutionary, or a next gen of internships.

But one thing that ALL crowdsourcing companies like to preach is how loyal and trustworthy and professional their community is. I know because I’ve read it in 100 different sites. Hell, I’ve written it a 100 different times here at uTest. So why do crowdsourcing companies insist upon telling the world how loyal and earnest their community is?  Maybe it’s to assuage the fears of prospective customers about entrusting their logo design, app development, content production or marketing to a community of strangers. Maybe it’s because if marketers say it enough times, we hope it’ll come true.

The more pessimistic view is that people — cloaked in the anonymity of the web — often act in greedy, selfish, mean-spirited ways (this perspective didn’t make it into the crowdsourcing brochure, by the way). Such dark behavior is well-documented and takes the form of flame wars on message boards, bullying via social media and online fraud.

So which is it — are people good-natured and honest?  Or are they money-hungry malcontents who will do anything to get ahead, as long as they don’t get caught?  Obviously it depends on the people, but I learned the truth about our community this week — and it was a lesson we learned the hard way.

A little background:  At uTest, we pay our testers twice per month via PayPal or Payoneer. And at this point in our growth, each pay cycle involves a non-trivial amount of cash — pretty deep into the five-figure range.  Now, it’s not easy or flattering to admit this, but in our most recent pay cycle, we experienced a glitch that caused us to pay our testers twice. That’s right folks, it was double payday here at uTest!

Read more…

Did you ever have that teacher who would give up on Friday afternoon and let the class watch a movie? We didn’t, hence your weekend reading assignment. What else are you gonna do while you wait in line for the iPad? That’s what I thought.

So, here are some testing/crowdsourcing articles to help pass the time. There will be a multiple choice exam on Monday. Or maybe we’ll just watch Ferris Bueller’s Day Off instead. Be prepared either way.

Serve Your Stakeholder
“Dr. Cem Kaner describes stakeholder as someone who has a vested interest in the success of the testing effort and/or in the success of the product. Stakeholders could be project managers, marketing managers, product managers, programmers, technical support representative, sales representatives, customers and many others in many different roles. Testing is done on behalf of the stakeholders. Here is a question for you. Have you ever asked who your stakeholder is before you start to test?” (From Parimala Shankaraiah’s latest post on curioustester.blogspot.com)

CNN Believes Whatever Computers Say
“Note to CNN: Information retrieved from an onboard computer is not reliable in cases where the computer itself is a suspect in the crime. No one is claiming that ghosts are causing the cars to go out of control. And there is no evidence yet that a mechanical failure is the culprit, despite the NHTSA looking hard for that evidence. Alcohol is also not a factor in these specific cases (or else we wouldn’t even be talking about them). That leaves two big possibilities: sober experienced people are suddenly forgetting how to drive OR something is wrong with the software or electronics that controls the cars.” (From James Bach’s blog on Satisfice.com)

Read more…

It’s back to the basics for the US Census. After several failed attempts to “modernize” the national headcount with hand held computers, officials are now blaming the resulting confusion, delays and costs on – wait for it – software bugs. You can read more about it here. To save you some time, I’ve summarized the progress made along the way with some telling quotations from those in the know:

Phase I: “Critical software errors are increasing; system performance is still lagging and testing continues to be compressed. A shortfall in testing portends potentially significant technical problems in the field.” – John Thompson, former associate director of the Census Bureau

Phase II: “The testing of the system is continuing to reveal critical defects.” – Todd Zinser, Commerce Department Inspector General

Phase III: “The performance of this system is not taking the load we’d like. It’s not going to accept the load that we will need to get it in about a month or so.” – Robert Groves, Census Director

Results: “They have prepared to do the follow-up using pencil and paper, and they’re very good at that.” – John Thompson (again)

Read more…

Well, today he’s sticking close to home in Boston. Tomorrow he’ll land in London… and before the week is out, he’ll hit Tel Aviv.

Doron starts Wednesday morning off (after his usual 10-mile run, of course!) in London with some tea and networking with friend and colleague, James Whittaker and UK partner, TCL.

Then he’s off to QCon London, an excellent conference for the enterprise software community. On Friday, 3/12 @ 2pm, he’ll be presenting at QCon re: The Mobile App Quality Challenge & How Crowdsourcing Can Help.

Doron is one of five software testing leaders chosen to present in the “How Do You Test That?” track. This track explores unique solutions created to address situations in which automated testing does not suffice.

And on the last leg of his marathon journey, Doron will present at Garage Geeks in Israel on Monday, 3/15 @ 8pm. There, Doron will be taking a deep dive into the topic of Crowdsourcing, and how smart recruiting, training and incentives can turn an unstructured, loosely assembled mob into a unified, professional community.

So, where in the world is Doron this week?  Catch him if you can!

VentureBeat has an interesting article about eBay’s announcement that they’re going to tap into their user base to test new features — a kind of opt-in, ongoing beta program for new features.  The title for this article:

eBay to Use Crowdsourcing to Test New Features, Starting with Streamlined Search

Those who know me well know that defending the purity of the term”crowdsourcing”  against misuse is a pet cause of mine (e.g. – Meet-ups are not crowdsourcing; online polls are not crowdsourcing; asking your Twitter followers a question is not crowdsourcing). But don’t worry… this won’t be another rant about the importance of definitions and how critical labels are.  Well, at least not about the word “crowdsourcing”.

Read more…

There’s no shame in being a couch potato (every now and again). After all,the competition between the major TV networks has been extremely entertaining. But now is the time to get up off the couch and onto your desktops, laptops and smartphones – it’s time for the very first Bug Battle of 2010!

That’s right, we’re pitting the four major TV networks against each other in this quarter’s bug-hunting competition. And while there are multiple sources to find out which network has the highest rated shows, we’re going to compare their web and mobile apps to see who distributes that content best.  Here are the networks, in alphabetical order: ABC, CBS, FOX, and NBC.

As an added twist, we’re introducing a mobile testing component to the Bug Battle. For more details on this – as well as info on rules and prizes – check out our Battle of the TV Networks page.

It’s now time to pull back the curtain to determine which network sites have the most bugs, the most critical bugs, and the best usability and feature set. And so uTesters, best of luck to you all as you begin another exciting bug hunting mission. May the best network win and may the top testers prevail. Good luck!

Special note: This competition is available only to members of the uTest community.  uTesters can get complete details and discuss the Bug Battle with their peers in the uTest Forums.

The major networks have been engaged in a decades-long struggle to win the hearts and minds (and eyes) of viewers. Whether it’s news, sports or sitcoms, these battles are now being fought on multiple fronts – including the distribution of their content through their web and mobile applications.

Which brings me to our latest bug-hunting competition. This quarter’s uTest Bug Battle will challenge testers to find bugs on the web AND mobile apps of the four major TV networks: CBS, NBC, ABC and FOX.

The competition gets started this Friday, February 5 at Noon (ET). At that time, testers from our community will be given one week to search these sites for the most compelling bugs, and to report them through our online platform. We’ll be dishing out nearly $4,000 in prize money for categories like Top Tester, Best Bug and Best Survey Feedback. As an added twist, we’re also including prizes for the Best Mobile Bugs.

More details on this Bug Battle – as well the rules, prizes and deadlines – after this commercial break. Just kidding, you can find them in the “Bug Battle” thread in the uTest Forums and we’ll be sharing more details here in the next few days.

So testers, you’ve got a few days to prepare for our most challenging competition to date. Until then, stay tuned.

You may have noticed that many top reporters and bloggers included “crowdsourcing” as a top trend of 2009. In fact, The New York Times named it 2009′s verb of the year!

But what will separate crowdsourcing’s winners and losers in 2010?  We believe that crowdsourcing success lies in a company’s ability to engage its community in unique and meaningful ways. These are the crowdsourcing companies that are changing the way work is done today — transforming professional services like software testing forever. But how does a company go beyond building a “mob” or a “crowd” and develop a community?

VOTE HERE to learn more about how crowdsourcing is changing the way work is done.

We’d love the opportunity to share our hard-earned keys to building a successful crowdsourcing business at this year’s Enterprise 2.0 Conference in Boston. But first, we need your help! (Very quick registration required.)

You can see a snapshot of uTest’s proposed presentation on the voting site. Voting opens today and closes on January 20th. As always, thanks for your continued support. All of our success stems from our amazing community!

Quick Update: If you’re posting online, #e2conf is the official tag! To track votes, click here.

Up there — it’s a bird… it’s a plane… Nope, it’s just the latest experiment to measure the ability of crowdsourcing to organize, mobilize, collaborate and compete.

DARPA (who along with Al Gore, invented the Internet), announced that it will place a large, red balloon at 10 different locations around the U.S.   The DARPA Network Challenge calls on groups to locate each of 10 red weather balloons scattered around the country — with $40,000 in prize money being awarded to the first team to accurately identify them all.

The purpose of this contest is to discover how social networking, crowdsourcing and other technologies can help accomplish a large-scale, time-critical task.  Already teams are forming, money is being spent, and the social media universe is abuzz about the competition, which launches on Saturday, December 5th.

Want to know more?  Check out what CNN, CNET, the Wall Street Journal and Scientific American have to say about the competition.

So if you see a big red orb tomorrow morning, call me!  And if nothing else, at least a Google search for the term “balloon” will return something other than another article on the plight of Balloon Boy!