Software Testing Blog

You’d be hard-pressed to find a more vocal proponent of crowdsourced software testing than yours truly (uTest). Over the last few years, we’ve seen first-hand just how successful community-based testing can be with regards to functional, security, load, localization and usability testing. But what about testing military systems? Could crowdsourced testing play a role in verifying the functionality of hi-tech weapons systems?

DARPA says yes.

Here with the story is informationweek.com:

The Department of Defense wants to create computer games that will crowdsource the complex process of verifying software for weapons systems.

The Defense Advanced Research Projects Agency (DARPA), the DOD’s research arm, through a project called Crowd Sourced Formal Verification (CSFV), aims to provide a “fun” way for the public to take part in software verification, a software engineering process to ensure an application satisfies its requirements, according to an agency announcement posted on FedBizOpps.gov.

The way DARPA sees it, if software verification was turned into a computer game that was fun for anyone to play, it could test the properties of software on a wider audience to ensure it will achieve its desired outcomes, according to the announcement.

Of course, one could argue that what DARPA is envisioning here is not really software testing as much as it is software checking – an important distinction made by many in the field, including James Bach. Nevertheless, there’s something to be said for the most advanced military on the planet looking to leverage the crowd for more accurate, more in-the-wild testing results. And if you recall, this is not DARPA’s first foray into crowdsourcing.

Read more…

In 2008, journalist Jeff Howe published Crowdsourcing: Why the Power of the Crowd Is Driving the Future of Business, in which he defined the term as follows:

“The act of taking a job traditionally performed by a designated agent (usually an employee) and outsourcing it to an undefined, generally large group of people in the form of an open call.”

Fairly straightforward, yes? Well, judging from the way the term is applied in modern media, there’s still some confusion over the true meaning crowdsourcing.

To illustrate, here are two recent news stories purporting to show crowdsourcing at work. One example fits the bill perfectly. The other does not. See if you can guess which is which…

Crowdsourced Video Gets Artistic With The Johnny Cash Project

Google Chrome – among all the other stuff they do – released a cool new video called The Johnny Cash Project, defined as a “collaborative art project attempting to stitch together a new music video for Johnny Cash’s “Ain’t No Grave” by having artists recreate individual frames and portraits of Cash in their own style.” The video was launched to mark the eight year anniversary of the singer’s death.

Here’s a video showing how it was made, followed by a brief description from reelseo.com:

Read more…

As you may have read on Monday’s blog post, uTest launched a new informational campaign to promote http://www.inthewildtesting.com.   The web site – and associated social media channels, including a Twitter profile – are intended to educate forward-thinking technology leaders about the necessity, benefits and real use cases of in-the-wild testing. 

We decided to launch it at TechCrunch Disrupt in San Francisco because the very concept of in-the-wild software testing (versus traditional methodologies) is, well…disruptive. 

Sure enough, TechCrunch Disrupt turned out to be the perfect event!  There were more than 2,600 innovative, entrepreneurial-minded techies, investors and exhibitors (35% more attendees than expected) filling the halls of the Design Concourse Center from Monday to Wednesday.  In its usual fashion, the conference itself attracted top industry-leaders such as Reid Hoffman of LinkedIn, Marissa Mayer of Google, Vinod Khosla, and even Ashton Kutcher.

uTest hosted a ton of terrific activities over the course of the event:

Read more…

Does this count as crowd sourcing or is it simply a test of the anonymous crowd’s integrity?  Either way, Jonathan’s “Starbucks card” is a very interesting examination of trust and expectation on the internet.

How to use the card:

1. Go to: http://jonathanstark.com/card/#get-a-coffee
2. Or save this picture (also embedded in the post) to your smartphone.
3. Hold the picture in front of the bar code scanner at checkout.
4. If the card has a positive balance, your caffeine is on Jonathan.

How to contribute to the card:

1. Visit starbucks.com/card
2. Click on the “Reload A Card” tab
3. Enter the card number visible in the picture (6061006913522430)
4. Click “Reload This Card” in the left sidebar
5. Choose a reload amount
6. Choose a payment method

Follow the card on Twitter @jonathanscard.

Comment below with your thoughts – especially if you’ve used or helped the card.

While we deal with crowdsourcing on the internet everyday, sometimes it’s easy to forget that some of the most difficult and complex logistical nightmares occur when you try to apply crowdsourcing to accomplish lofty goals in the physical world. However, when leveraged correctly, crowdsourcing is a powerful tool. And like any powerful tool, it can be used for good and evil. With great power comes great… well you get the picture.

Today we’re looking at some of more interesting “real life” crowdsourcing endeavors in the past few years.

The Good
While the world watched and read about the riots in London, most people were caught up with why they were occurring, and the devastation the riots were causing. However the morning after the pandemonium, hundreds of London citizens gathered in effort to clean up their city. This effort started on Twitter while the riots were still underway. While many news outlets and tech sites focused on the role of Blackberry Messenger and Twitter in helping spread the carnage, some of the same technology was instrumental citizens from the riots.

The Bad
Now this one was not nearly as large scale as the previous story, but made the internet news rounds just the same when it happened back in 2008. A man used Craigslist enlisting an unwitting crowd as decoys to provide cover for him to rob an armored car and escape. Even today, strange ads are not uncommon on Craigslist, and while a few have landed the company in lawsuits, this particular posting didn’t raise flags until after the robbery took place.

Read more…

Working at a start-up is kind of like having kids. You get excited about every new milestone like a proud parent, even though you know it’s not the first time any of it has ever happened in history. This time it’s yours. Your child. Your company.

For parents, new milestones usually mean posting a select, few pictures to Facebook (give or take 50) of an an event like your child sitting up for the first time. Perched like a top-heavy teddy bear, staring wide-eyed at the camera and being photographed with adoration from many, many…many angles.

For uTest, it usually means hitting the local pub and celebrating together with a pint or two (give or take).

But this milestone was special. Our company, headquartered outside of Boston, opened up our first office in Silicon Valley. Exciting stuff for our founding members and newcomers alike! Granted, with so many recurring clients out here in the Bay Area, uTest has had a West Coast team for over a year, incubating in the Sunnyvale Plug & Play Tech Center.

But to keep up with demand, we’ve expanded pretty fast and definitely outgrew the nest. So now we have a space of our own in bustling downtown San Mateo, one block from the Cal Train and smack dab in the middle of a ton of great restaurants and shops.

What better way to celebrate the milestone and break in our new digs than throw a party! With our CEO Doron Reuveni and the executive team in town, that’s exactly what we did.

Living by the quote of our CMO Matt Johnston– “We take our work seriously, but not ourselves”– we spent a warm summer night celebrating with about 75 of our investors, customers, partners, new neighbors (the good folks at Attributor), new and old friends in the industry, the Chamber of Commerce…and even Mayor Jack Matthews.  Sushi, drinks, and music in the company of some of the Valley’s smartest, funniest, and most fascinating people. It doesn’t get better than this.

In fact, we’re already thinking about throwing another party in a few months for the many who couldn’t make it to this one.  Who knows…we may have a whole new milestone to celebrate.  And cleared space on our camera.

Just in case crowdsourcing needed more reinforcement of its value, the country of Iceland has turned to the population to help write its next constitution.

As the Guardian writes:

In creating the new document, the council has been posting draft clauses on its website every week since the project launched in April. The public can comment underneath or join a discussion on the council’s Facebook page.

The council also has a Twitter account, a YouTube page where interviews with its members are regularly posted, and a Flickr account containing pictures of the 25 members at work, all intended to maximise interaction with citizens.

Meetings of the council are open to the public and streamed live on to the website and Facebook page. The latter has more than 1,300 likes in a country of 320,000 people.

It will be interesting to hear about the process after it’s completed.

For those interested, we have a white paper discussing crowdsourcing as it relates to software testing.

We’re kicking off a new, local chapter of the Crowdsortium — the most fascinating and inspiring group of crowdsourcing thought leaders and pundits that have self-organized to advance the concept of leveraging online communities to get work done.

The first Crowdsortium Meetup in Cambridge, MA will be hosted by Appswell and yours truly at the Microsoft NERD, June 21st at 6:30pm.

We have quite an exciting line up to offer you! Let’s start with the keynote: Harvard professor and leading academic expert on crowdsourcing, Karim Lakhani will open the meetup.

The feature panel — “What’s New & What’s Next in Crowdsourcing” — is up next featuring local VCs, CEOs and thought leaders of top crowdsourcing companies…

• Jeff Howe, Contributing Editor at Wired Magazine & Author of Crowdsourcing
• Dwayne Spradlin, CEO of Innocentive
• Doron Reuveni, CEO of uTest
• Daniel Sullivan, CEO of Appswell
• Jim Savage (moderator), Partner and Co-Founder of Longworth Venture Partners

These industry leaders have had great success advancing the ever-changing and growing crowdsourcing industry and will dispel some of the most common misconceptions about crowdsourcing and reveal their secrets on how to make it work.

Last, we will close it all up with networking along with FREE PIZZA AND BEER! RSVP to reserve your spot here (free to attend).

In 2009, an ambitious campaign to purchase Pabst Brewing Company was launched by executives at two advertising agencies. Brian William Flatow and Michael Migliozzi II of The Ad Store & Forza Migliozzi respectively, came up with “an interesting experiment in crowdsourcing.”

The duo intended to bring together fans in order to raise $300 million make the purchase of Pabst possible. The concept was simple. Using social media sites like Twitter and Facebook to get the word out, Pabst fans (of legal age) could visit BuyaBeerCompany.com and pledge between $5 and $250,000 towards the acquisition. Investors were not required to transfer funds until the goal of $300 million was reached.

The campaign generated an overwhelming response, gathering about 5 million fans who pledged to invest a combined $200 million at its height. However due to a major oversight, the federal government halted the venture. Flatow and Migliozzi failed to register the public offering with the SEC before they started to sell shares to the public, a violation of federal law. The Securities and Exchange Commission reached a settlement with the two ad execs this week and the pair agreed to stop sales.

The dynamic nature of crowdsourcing allows it to be utilized in new and innovative ways, but this presents new legal challenges to companies and regulating bodies. The attorney for the two ad execs was quoted as saying “it never dawned on them” that they needed to register the offering without any shares being sold.

Read more…

As we get closer to reaching the critical milestone of 40,000 testers in the uTest community (any day now!), we knew we’d have to find ways to scale our community programs in order to manage, vet and engage our enormous pool of expert testers and QA professionals.

Well, we didn’t have to look very far! The answers were right in front of us — where else but within the uTester community itself. As Matt said, “We found inspiration in seeing the enthusiasm and personal pride that resulted from members advising, supporting and training other members in our tester forums.”

From there, we launched three new initiatives – Sandbox, Crash Courses, and Test Team Lead – which catapulted uTest into the next phase of crowdsourcing, one that is self-sufficient, self-teaching, and self-policing.

One that is “For uTesters, By uTesters.”

For example, the Test Team Lead program gives uTest members the opportunity to earn paid leadership roles, mentoring and helping other testers succeed in their work. Crash Courses provide community-generated training materials to raise the skill level of community members and their eligibility to accept more work. More than 50 Crash Courses have been written to advance tester skills and further their careers.

Check out some early thoughts from the media on TechTarget, Crowdsourcing.org, TheDailyCrowdsource.

Details on each program after the bump!

Read more…