DARPA says yes.

Here with the story is informationweek.com:

The Department of Defense wants to create computer games that will crowdsource the complex process of verifying software for weapons systems.

The Defense Advanced Research Projects Agency (DARPA), the DOD’s research arm, through a project called Crowd Sourced Formal Verification (CSFV), aims to provide a “fun” way for the public to take part in software verification, a software engineering process to ensure an application satisfies its requirements, according to an agency announcement posted on FedBizOpps.gov.

The way DARPA sees it, if software verification was turned into a computer game that was fun for anyone to play, it could test the properties of software on a wider audience to ensure it will achieve its desired outcomes, according to the announcement.

Of course, one could argue that what DARPA is envisioning here is not really software testing as much as it is software checking – an important distinction made by many in the field, including James Bach. Nevertheless, there’s something to be said for the most advanced military on the planet looking to leverage the crowd for more accurate, more in-the-wild testing results. And if you recall, this is not DARPA’s first foray into crowdsourcing.

The bigger issue here is what, exactly, the crowd will be testing and what these “desired outcomes” will be. Let’s hope it goes better than this….

“The act of taking a job traditionally performed by a designated agent (usually an employee) and outsourcing it to an undefined, generally large group of people in the form of an open call.”

Fairly straightforward, yes? Well, judging from the way the term is applied in modern media, there’s still some confusion over the true meaning crowdsourcing.

To illustrate, here are two recent news stories purporting to show crowdsourcing at work. One example fits the bill perfectly. The other does not. See if you can guess which is which…

Crowdsourced Video Gets Artistic With The Johnny Cash Project

Google Chrome – among all the other stuff they do – released a cool new video called The Johnny Cash Project, defined as a “collaborative art project attempting to stitch together a new music video for Johnny Cash’s “Ain’t No Grave” by having artists recreate individual frames and portraits of Cash in their own style.” The video was launched to mark the eight year anniversary of the singer’s death.

Here’s a video showing how it was made, followed by a brief description from reelseo.com:

“This is crowdsourcing at its finest, and the resulting collaborative video is compelling. Also, because there are so many great works of art represented in the piece, it rewards multiple viewings. It’s interesting to any fan of music and art, but for Cash fans… I imagine its impact is even more profound.

The entire thing is created by the masses. The idea came from Director Chris Milk and a host of his producers and editors, but the actual content comes from Johnny Cash and a quarter of a million Internet users. Every time I think I grasp the draw and the power of online video, something like this comes along to remind me we’ve only scratched the surface.”

Notice a few key phrases which bear the mark of crowdsourcing: created by the masses, multiple users, peer-to-peer collaboration, etc.

Now take a look at this “crowdsourcing” story from The Chicago Tribune.

Crowd-sourcing the budget

“The new mayor asked Chicagoans for ideas on how to balance the budget, and he got thousands of them. Cut the electric bill by turning off every other streetlight. Make out-of-town motorists pay a toll to enter the city. Install a zip line and charge thrill-seekers to surf between skyscrapers. Those were among our favorites, printed last week on the Perspective page.”

See the difference? In the second example, we find what equates to an online poll or message board. No collaboration, no problem-solving, no creation – just a soapbox for thrifty readers and voters (although I dig the idea of surfing between skyscrapers).

Anyway, I wanted to highlight this misconception using specific, real-world examples. Partly because I needed an excuse to post Johnny Cash to our blog, but mostly because I wanted to reinforce the idea that just because someone calls it crowdsourcing, doesn’t make it so.

We decided to launch it at TechCrunch Disrupt in San Francisco because the very concept of in-the-wild software testing (versus traditional methodologies) is, well…disruptive. 

Sure enough, TechCrunch Disrupt turned out to be the perfect event!  There were more than 2,600 innovative, entrepreneurial-minded techies, investors and exhibitors (35% more attendees than expected) filling the halls of the Design Concourse Center from Monday to Wednesday.  In its usual fashion, the conference itself attracted top industry-leaders such as Reid Hoffman of LinkedIn, Marissa Mayer of Google, Vinod Khosla, and even Ashton Kutcher.

uTest hosted a ton of terrific activities over the course of the event:

On Sunday, we spent the day at the Disrupt Hackathon, selecting a winner for uTest’s “Most Likely to Succeed in the Wild” award, which came with a prize worth $3,000 of uTest’s services.  Wow- what a day!  More than 700 participants worked overnight and coded a record number (130!) of hacks. We watched as each of the teams mustered the energy to climb on-stage and pitch their project in the allotted 60 seconds.  The sheer breadth of creativity was impressive.  Yet, we had to make a decision, so we chose the masterminds behind AirCart, which enables grocery store self-checkout through your phone, rather than the standard dedicated checkout stand. (See pic to the right)

On Monday and Tuesday evenings, we had a special surprise for Disrupt.  In order to promote in-the-wild testing, we sponsored the urban Serengeti-themed “In the Wild Oasis” RVIP, a mobile karaoke lounge created in a renovated Winnebago.  Sounds crazy and fun?  It was!  (The RVIP is the brainchild of Kestrin Pantera, who by the way, sounds dead-on like Susan Sarandon’s ‘Janet’ when she karaokes Rocky Horror.)

During Happy Hour, we were parked outside the front of the Concourse Center, where people climbed aboard, checked it out, had a drink, and started relaxing from the day.  From there, the RV– blanketed in lights and glowing like Chevy Chase’s house in Christmas Vacation– took to the ‘wilds’ of SF, providing shuttle service for folks to their hotels. But mostly getting a whole lot of smiles and laughter from people on the streets.

By 9:30 both nights, we’d arrived at the official TechCrunch after parties at 1015 Folsom (sponsored by Bridgewater) and Roe (sponsored by MailChimp).  Parked there, we partied through the night…usually until 2:00am.  Throughout the two nights, hundreds of Disrupters had climbed aboard, to have a drink, chat and network in the RV’s lounge.  Some groups had stuffed themselves into the photo booth for pics. 

And almost everyone karaoked.  How could they not?  When the entire RV is rocking from the dancing, and the whole crowd is singing Wild Thing, Hotel California or Rock Lobster…it’s impossible to resist!  Even TechCrunch reporter Jason Kincaid hung out and belted some tunes (btw: He has a great voice…who knew?).

As we’ve been known to say at uTest, “We take our jobs seriously.  But not ourselves.”  Disrupt was a great example.  People learned about in-the-wild testing and our business ideas, and we learned about theirs.  We connected with customers, old friends and partners, and made new ones.  And we all had a blazing good time.

Until next year…!

For many more photos, visit the TC Disrupt page on www.inthewildtesting.com or uTest’s Flickr page.

How to use the card:

1. Go to: http://jonathanstark.com/card/#get-a-coffee
2. Or save this picture (also embedded in the post) to your smartphone.
3. Hold the picture in front of the bar code scanner at checkout.
4. If the card has a positive balance, your caffeine is on Jonathan.

How to contribute to the card:

1. Visit starbucks.com/card
2. Click on the “Reload A Card” tab
3. Enter the card number visible in the picture (6061006913522430)
4. Click “Reload This Card” in the left sidebar
5. Choose a reload amount
6. Choose a payment method

Follow the card on Twitter @jonathanscard.

Comment below with your thoughts – especially if you’ve used or helped the card.

Today we’re looking at some of more interesting “real life” crowdsourcing endeavors in the past few years.

The Good
While the world watched and read about the riots in London, most people were caught up with why they were occurring, and the devastation the riots were causing. However the morning after the pandemonium, hundreds of London citizens gathered in effort to clean up their city. This effort started on Twitter while the riots were still underway. While many news outlets and tech sites focused on the role of Blackberry Messenger and Twitter in helping spread the carnage, some of the same technology was instrumental citizens from the riots.

The Bad
Now this one was not nearly as large scale as the previous story, but made the internet news rounds just the same when it happened back in 2008. A man used Craigslist enlisting an unwitting crowd as decoys to provide cover for him to rob an armored car and escape. Even today, strange ads are not uncommon on Craigslist, and while a few have landed the company in lawsuits, this particular posting didn’t raise flags until after the robbery took place.

The ad offered construction work in Monroe, WA, a city northeast of Seattle. About a dozen men replied to the ad and they were all instructed meet near a Bank of America wearing a yellow vest, safety goggles, respirator mask and a blue shirt. While they gathered, the assailant, wearing the same gear pepper sprayed a guard transporting canvas bags filled with cash to the bank and escaped about 100 yards on foot with the cash to a nearby creek where he jumped into an inner tube to complete his getaway.

The Ugly
Then there is the flash mob, the dance variety in particular. Sometimes dance flash mobs are cute and delight onlookers. For instance, when a seemingly distracted crowd of 500 kids dance in unison at a Sacramento city park as part of an after school program. Of course then there is the flash mob that’s made up of a thousand neon-clad Zumba enthusiasts. The fitness enthusiasts held an impromptu workout party for “The Today Show” in NY. The show expected a huge turnout, but the mob grew beyond anyone’s expectations.

There was no doubt it exposed many viewers to the popularity of the Latin dance inspired fitness program, but also how excruciating it is to the eyes when over a thousand people gather all wearing bright neon.

What other examples of crowdsourcing – both the good and the bad – come to mind?

For parents, new milestones usually mean posting a select, few pictures to Facebook (give or take 50) of an an event like your child sitting up for the first time. Perched like a top-heavy teddy bear, staring wide-eyed at the camera and being photographed with adoration from many, many…many angles.

For uTest, it usually means hitting the local pub and celebrating together with a pint or two (give or take).

But this milestone was special. Our company, headquartered outside of Boston, opened up our first office in Silicon Valley. Exciting stuff for our founding members and newcomers alike! Granted, with so many recurring clients out here in the Bay Area, uTest has had a West Coast team for over a year, incubating in the Sunnyvale Plug & Play Tech Center.

But to keep up with demand, we’ve expanded pretty fast and definitely outgrew the nest. So now we have a space of our own in bustling downtown San Mateo, one block from the Cal Train and smack dab in the middle of a ton of great restaurants and shops.

What better way to celebrate the milestone and break in our new digs than throw a party! With our CEO Doron Reuveni and the executive team in town, that’s exactly what we did.

Living by the quote of our CMO Matt Johnston– “We take our work seriously, but not ourselves”– we spent a warm summer night celebrating with about 75 of our investors, customers, partners, new neighbors (the good folks at Attributor), new and old friends in the industry, the Chamber of Commerce…and even Mayor Jack Matthews.  Sushi, drinks, and music in the company of some of the Valley’s smartest, funniest, and most fascinating people. It doesn’t get better than this.

In fact, we’re already thinking about throwing another party in a few months for the many who couldn’t make it to this one.  Who knows…we may have a whole new milestone to celebrate.  And cleared space on our camera.

As the Guardian writes:

In creating the new document, the council has been posting draft clauses on its website every week since the project launched in April. The public can comment underneath or join a discussion on the council’s Facebook page.

The council also has a Twitter account, a YouTube page where interviews with its members are regularly posted, and a Flickr account containing pictures of the 25 members at work, all intended to maximise interaction with citizens.

Meetings of the council are open to the public and streamed live on to the website and Facebook page. The latter has more than 1,300 likes in a country of 320,000 people.

It will be interesting to hear about the process after it’s completed.

For those interested, we have a white paper discussing crowdsourcing as it relates to software testing.

The first Crowdsortium Meetup in Cambridge, MA will be hosted by Appswell and yours truly at the Microsoft NERD, June 21st at 6:30pm.

We have quite an exciting line up to offer you! Let’s start with the keynote: Harvard professor and leading academic expert on crowdsourcing, Karim Lakhani will open the meetup.

The feature panel — “What’s New & What’s Next in Crowdsourcing” — is up next featuring local VCs, CEOs and thought leaders of top crowdsourcing companies…

• Jeff Howe, Contributing Editor at Wired Magazine & Author of Crowdsourcing
• Dwayne Spradlin, CEO of Innocentive
• Doron Reuveni, CEO of uTest
• Daniel Sullivan, CEO of Appswell
• Jim Savage (moderator), Partner and Co-Founder of Longworth Venture Partners

These industry leaders have had great success advancing the ever-changing and growing crowdsourcing industry and will dispel some of the most common misconceptions about crowdsourcing and reveal their secrets on how to make it work.

Last, we will close it all up with networking along with FREE PIZZA AND BEER! RSVP to reserve your spot here (free to attend).

The duo intended to bring together fans in order to raise $300 million make the purchase of Pabst possible. The concept was simple. Using social media sites like Twitter and Facebook to get the word out, Pabst fans (of legal age) could visit BuyaBeerCompany.com and pledge between $5 and $250,000 towards the acquisition. Investors were not required to transfer funds until the goal of $300 million was reached.

The campaign generated an overwhelming response, gathering about 5 million fans who pledged to invest a combined $200 million at its height. However due to a major oversight, the federal government halted the venture. Flatow and Migliozzi failed to register the public offering with the SEC before they started to sell shares to the public, a violation of federal law. The Securities and Exchange Commission reached a settlement with the two ad execs this week and the pair agreed to stop sales.

The dynamic nature of crowdsourcing allows it to be utilized in new and innovative ways, but this presents new legal challenges to companies and regulating bodies. The attorney for the two ad execs was quoted as saying “it never dawned on them” that they needed to register the offering without any shares being sold.

This situation shines a light on the fact that companies taking advantage of crowdsourcing are not only blazing trails in their respective industries, but may also be treading into new territory when it comes to legal matters. Flatow or Migliozzi neither admitted nor denied any wrongdoing when they complied with the SEC’s cease-and-desist order. Pabst Brewing Company was sold to C. Dean Metropoulos, a food industry executive, in June 2010.

Another reminder that not all crowdsourcing ventures can be as successful as uTest

Well, we didn’t have to look very far! The answers were right in front of us — where else but within the uTester community itself. As Matt said, “We found inspiration in seeing the enthusiasm and personal pride that resulted from members advising, supporting and training other members in our tester forums.”

From there, we launched three new initiatives – Sandbox, Crash Courses, and Test Team Lead – which catapulted uTest into the next phase of crowdsourcing, one that is self-sufficient, self-teaching, and self-policing.

One that is “For uTesters, By uTesters.”

For example, the Test Team Lead program gives uTest members the opportunity to earn paid leadership roles, mentoring and helping other testers succeed in their work. Crash Courses provide community-generated training materials to raise the skill level of community members and their eligibility to accept more work. More than 50 Crash Courses have been written to advance tester skills and further their careers.

Check out some early thoughts from the media on TechTarget, Crowdsourcing.org, TheDailyCrowdsource.

Details on each program after the bump!

About uTest Sandbox
With nearly 1,000 new testers joining uTest every month, the company developed the uTest Sandbox program, a simulated testing project that enables uTest to evaluate the skills and quality of new community members. Here’s how it works:

• When a tester’s profile information matches current or expected project demand, that tester is automatically invited to a weekly Sandbox project.
• Each Sandbox test cycle mimics a real testing project with a target website to test, instructions to follow, and bugs and test cases to submit.
• Each Sandbox project is evaluated by top-performing veteran uTesters, called Sandbox Team Leads (see more about this program below). These proven performers vet new uTesters based on ability to follow instructions, uncover valuable bugs, write clear bug reports, and include relevant documentation such as screenshots and video captures.
• Sandbox graduates with the highest grades, as determined by the Test Team Lead, will begin building their uTest rating, and will be placed at the front of the queue to receive upcoming paid testing opportunities.

About uTest Crash Courses
With nearly 40,000 testers in its community, uTest discovered the global appetite for testing-related training to be monumental. Members have expressed interest in learning about a wide variety of software testing specialties, from basic functional testing of iPhone or Android apps to more advanced training such as performing security audits or performance engineering.

To meet this need, uTest launched the Crash Course program, which follows a FUBU formula (“For uTesters, By uTesters”) to create real-world training materials. Here’s how it works:

• Any uTester can request a Crash Course on any topic– from broad subjects like testing web apps to highly-focused subjects such as performing usability testing on iPad apps.
• Then the topics get voted on by uTest community members. If there’s enough interest in a particular topic, uTest will identify testers with the most experience and success in that specific area, based on their performance in related projects.
• uTest will then approach the member and inquire into his interest in producing the Crash Course, providing him esteem within the community as a subject matter expert.
• During this program’s brief beta period, more than 50 Crash Courses were written and remain freely available to all uTesters to advance their skills and further their careers.

About Test Team Lead
Test Team Lead is a program that gives uTest members the opportunity to assume paid leadership roles within the community, mentoring and helping other testers succeed in their work. To succeed, a member must demonstrate skill in effectively managing other people, work well with uTest project managers, and provide high-quality customer interaction.

There are two ways members can participate: Test Team Project Leads or Sandbox Team Leads. Members are selected based on their past performance on uTest projects, uTest’s experience working with them, and their organizational and leadership skills.

Test Team Sandbox Leads moderate the Sandbox training and serve as role models for the uTest community.  During Sandbox training sessions, they are responsible for:

• Role playing from the perspective of uTest project manager and/or customer to evaluate tester interaction and performance
• Grading and recommending testers based on an agreed upon grading rubric

Test Team Project Leads are graduates of the Sandbox Team Lead program. In this more advanced role, the member supports uTest’s project managers on customer-facing projects by:

• Communicating with testers when more reporting is required
• Answering testers’ questions about projects and applications
• Reproducing submitted bugs for quality control
• Adding new and valuable information about discovered bugs as needed

If you have any questions, please feel free to contact us at marketing@utest.com.