Software Testing Blog

You’d be hard-pressed to find a more vocal proponent of crowdsourced software testing than yours truly (uTest). Over the last few years, we’ve seen first-hand just how successful community-based testing can be with regards to functional, security, load, localization and usability testing. But what about testing military systems? Could crowdsourced testing play a role in verifying the functionality of hi-tech weapons systems?

DARPA says yes.

Here with the story is informationweek.com:

The Department of Defense wants to create computer games that will crowdsource the complex process of verifying software for weapons systems.

The Defense Advanced Research Projects Agency (DARPA), the DOD’s research arm, through a project called Crowd Sourced Formal Verification (CSFV), aims to provide a “fun” way for the public to take part in software verification, a software engineering process to ensure an application satisfies its requirements, according to an agency announcement posted on FedBizOpps.gov.

The way DARPA sees it, if software verification was turned into a computer game that was fun for anyone to play, it could test the properties of software on a wider audience to ensure it will achieve its desired outcomes, according to the announcement.

Of course, one could argue that what DARPA is envisioning here is not really software testing as much as it is software checking – an important distinction made by many in the field, including James Bach. Nevertheless, there’s something to be said for the most advanced military on the planet looking to leverage the crowd for more accurate, more in-the-wild testing results. And if you recall, this is not DARPA’s first foray into crowdsourcing.

Read more…

First off the title of this post will require a bit of explanation. W.M.D. in this context stands for “World of Mass Development,” which is a new platform for games creation from the award-winning games developer Slightly Mad Studios. Not only does this new platform crowdsource ideas and designs for games among the developer/designer communities, but they also expand the crowdsourced community to include investors. W.M.D.‘s first title will be a racing game called “C.A.R.S.” which stands for “Community Assisted Racing Simulation”.

Members of the public may pay a subscription fee to the community and help develop the game, and even possibly get a return on their investment once the game ships. Those who participate will be able to play the game from its pre-alpha builds up to its final release version. Slightly Mad plans to take 30% of the profits, with the other 70% being divided among the community investors.

“Traditional development puts developers at the mercy of publishers,” Slightly Mad said. “The development process offered by WMD shifts the focus back to creating great games that your target audience wants to play, whilst still offering the chance to get proper funding for development and testing.”

Read more…

In 2008, journalist Jeff Howe published Crowdsourcing: Why the Power of the Crowd Is Driving the Future of Business, in which he defined the term as follows:

“The act of taking a job traditionally performed by a designated agent (usually an employee) and outsourcing it to an undefined, generally large group of people in the form of an open call.”

Fairly straightforward, yes? Well, judging from the way the term is applied in modern media, there’s still some confusion over the true meaning crowdsourcing.

To illustrate, here are two recent news stories purporting to show crowdsourcing at work. One example fits the bill perfectly. The other does not. See if you can guess which is which…

Crowdsourced Video Gets Artistic With The Johnny Cash Project

Google Chrome – among all the other stuff they do – released a cool new video called The Johnny Cash Project, defined as a “collaborative art project attempting to stitch together a new music video for Johnny Cash’s “Ain’t No Grave” by having artists recreate individual frames and portraits of Cash in their own style.” The video was launched to mark the eight year anniversary of the singer’s death.

Here’s a video showing how it was made, followed by a brief description from reelseo.com:

Read more…

Black Friday is right around the corner, that’s right folks the season of giving (but mostly receiving) will be here soon. Before we get there though, Target unexpectedly gave its online shoppers a little taste of the madness before the leaves have even changed color.

So what was this frenzy over? Have Pogs crawled their way back into children’s hearts? Has the Tamagotchi reclaimed the title of most ridiculous toy of all time? Nope. Popular Italian designer Missoni released a limited 400 piece collection of  cheap designer clothing and accessories exclusively at Target.

Target heavily marketed this campaign via Twitter and other social media. The result? Crickets. Not because no one wanted the merchandise, quite the contrary actually, people were after it something fierce. The issue? Due to the ridiculous strain of millions of people hitting Target’s website at the same time, it crashed. People were quick to take to Twitter to let Target know how they felt, naturally many were quite disappointed when the few that could get in gobbled up all the merchandise in a matter of hours.

Now I like Bullseye as much as the next guy, but to have to stare at him all day instead of buying my Missoni woven zigzag scarf? Totally bogus.

So how should companies test for load and performance? To summarize our Marketing Maestro Stanton Champion’s article on load testing, companies should take a gander at the following load planning tips before unleashing their site to frantic fashionistas:

Read more…

Does this count as crowd sourcing or is it simply a test of the anonymous crowd’s integrity?  Either way, Jonathan’s “Starbucks card” is a very interesting examination of trust and expectation on the internet.

How to use the card:

1. Go to: http://jonathanstark.com/card/#get-a-coffee
2. Or save this picture (also embedded in the post) to your smartphone.
3. Hold the picture in front of the bar code scanner at checkout.
4. If the card has a positive balance, your caffeine is on Jonathan.

How to contribute to the card:

1. Visit starbucks.com/card
2. Click on the “Reload A Card” tab
3. Enter the card number visible in the picture (6061006913522430)
4. Click “Reload This Card” in the left sidebar
5. Choose a reload amount
6. Choose a payment method

Follow the card on Twitter @jonathanscard.

Comment below with your thoughts – especially if you’ve used or helped the card.

While we deal with crowdsourcing on the internet everyday, sometimes it’s easy to forget that some of the most difficult and complex logistical nightmares occur when you try to apply crowdsourcing to accomplish lofty goals in the physical world. However, when leveraged correctly, crowdsourcing is a powerful tool. And like any powerful tool, it can be used for good and evil. With great power comes great… well you get the picture.

Today we’re looking at some of more interesting “real life” crowdsourcing endeavors in the past few years.

The Good
While the world watched and read about the riots in London, most people were caught up with why they were occurring, and the devastation the riots were causing. However the morning after the pandemonium, hundreds of London citizens gathered in effort to clean up their city. This effort started on Twitter while the riots were still underway. While many news outlets and tech sites focused on the role of Blackberry Messenger and Twitter in helping spread the carnage, some of the same technology was instrumental citizens from the riots.

The Bad
Now this one was not nearly as large scale as the previous story, but made the internet news rounds just the same when it happened back in 2008. A man used Craigslist enlisting an unwitting crowd as decoys to provide cover for him to rob an armored car and escape. Even today, strange ads are not uncommon on Craigslist, and while a few have landed the company in lawsuits, this particular posting didn’t raise flags until after the robbery took place.

Read more…

Just in case crowdsourcing needed more reinforcement of its value, the country of Iceland has turned to the population to help write its next constitution.

As the Guardian writes:

In creating the new document, the council has been posting draft clauses on its website every week since the project launched in April. The public can comment underneath or join a discussion on the council’s Facebook page.

The council also has a Twitter account, a YouTube page where interviews with its members are regularly posted, and a Flickr account containing pictures of the 25 members at work, all intended to maximise interaction with citizens.

Meetings of the council are open to the public and streamed live on to the website and Facebook page. The latter has more than 1,300 likes in a country of 320,000 people.

It will be interesting to hear about the process after it’s completed.

For those interested, we have a white paper discussing crowdsourcing as it relates to software testing.

We’re kicking off a new, local chapter of the Crowdsortium — the most fascinating and inspiring group of crowdsourcing thought leaders and pundits that have self-organized to advance the concept of leveraging online communities to get work done.

The first Crowdsortium Meetup in Cambridge, MA will be hosted by Appswell and yours truly at the Microsoft NERD, June 21st at 6:30pm.

We have quite an exciting line up to offer you! Let’s start with the keynote: Harvard professor and leading academic expert on crowdsourcing, Karim Lakhani will open the meetup.

The feature panel — “What’s New & What’s Next in Crowdsourcing” — is up next featuring local VCs, CEOs and thought leaders of top crowdsourcing companies…

• Jeff Howe, Contributing Editor at Wired Magazine & Author of Crowdsourcing
• Dwayne Spradlin, CEO of Innocentive
• Doron Reuveni, CEO of uTest
• Daniel Sullivan, CEO of Appswell
• Jim Savage (moderator), Partner and Co-Founder of Longworth Venture Partners

These industry leaders have had great success advancing the ever-changing and growing crowdsourcing industry and will dispel some of the most common misconceptions about crowdsourcing and reveal their secrets on how to make it work.

Last, we will close it all up with networking along with FREE PIZZA AND BEER! RSVP to reserve your spot here (free to attend).

In 2009, an ambitious campaign to purchase Pabst Brewing Company was launched by executives at two advertising agencies. Brian William Flatow and Michael Migliozzi II of The Ad Store & Forza Migliozzi respectively, came up with “an interesting experiment in crowdsourcing.”

The duo intended to bring together fans in order to raise $300 million make the purchase of Pabst possible. The concept was simple. Using social media sites like Twitter and Facebook to get the word out, Pabst fans (of legal age) could visit BuyaBeerCompany.com and pledge between $5 and $250,000 towards the acquisition. Investors were not required to transfer funds until the goal of $300 million was reached.

The campaign generated an overwhelming response, gathering about 5 million fans who pledged to invest a combined $200 million at its height. However due to a major oversight, the federal government halted the venture. Flatow and Migliozzi failed to register the public offering with the SEC before they started to sell shares to the public, a violation of federal law. The Securities and Exchange Commission reached a settlement with the two ad execs this week and the pair agreed to stop sales.

The dynamic nature of crowdsourcing allows it to be utilized in new and innovative ways, but this presents new legal challenges to companies and regulating bodies. The attorney for the two ad execs was quoted as saying “it never dawned on them” that they needed to register the offering without any shares being sold.

Read more…

As we get closer to reaching the critical milestone of 40,000 testers in the uTest community (any day now!), we knew we’d have to find ways to scale our community programs in order to manage, vet and engage our enormous pool of expert testers and QA professionals.

Well, we didn’t have to look very far! The answers were right in front of us — where else but within the uTester community itself. As Matt said, “We found inspiration in seeing the enthusiasm and personal pride that resulted from members advising, supporting and training other members in our tester forums.”

From there, we launched three new initiatives – Sandbox, Crash Courses, and Test Team Lead – which catapulted uTest into the next phase of crowdsourcing, one that is self-sufficient, self-teaching, and self-policing.

One that is “For uTesters, By uTesters.”

For example, the Test Team Lead program gives uTest members the opportunity to earn paid leadership roles, mentoring and helping other testers succeed in their work. Crash Courses provide community-generated training materials to raise the skill level of community members and their eligibility to accept more work. More than 50 Crash Courses have been written to advance tester skills and further their careers.

Check out some early thoughts from the media on TechTarget, Crowdsourcing.org, TheDailyCrowdsource.

Details on each program after the bump!

Read more…