Testing the Limits with James Whittaker – Part II

In the second part of our Testing the Limits interview with James Whittaker, we tackle Google vs. Microsoft; dogs vs. cats; why SCRUM is just a name; his advice for college graduates; bad habits of exploratory testing and more. If you missed Part I, you can find it here.

If you want to read more of James’ work, bookmarking the Google Testing Blog would be a good place to start. You can also read his 2009 book Exploratory Software Testing or check out some of his uTest eBooks and webinars.

uTest: The Microsoft vs. Google battle continues to play out very publicly in the media. Just last week, Computerworld wrote this story: “Microsoft: No Matter What Google Says, Windows Is Secure.” Having been at both companies, we think you have a unique perspective on this one. Any thoughts?

JW: Let me say right away that I enjoyed my time at Microsoft and admire the company and the smart people who work there. As a resident of Seattle, it is in my best interest for Microsoft to prosper! But the two companies are vastly different regarding the way their talent is managed and their products are built. Google is an engineering-centric company where innovation comes from individuals who are empowered to do whatever they need to get ideas into production. Much has been made of Google’s game-theory approach to managing people where rewards are given quickly for impactful behavior. It works. Morale is high and people work very hard and take quality very seriously.

Does this mean we produce more secure or more reliable products? We try hard to do so; Microsoft tries equally hard. I think we have the advantage of less legacy and a more modern and reliable platform (the Web as opposed to client operating systems) to work from. But the secret sauce at both companies is the same: hard work and due diligence.

uTest: You shared with us (as the pioneer of Testing the Limits posts) that your first assignment at Google was “To raise the level of testing precision and diligence.” So, how did it go?

JW: It didn’t take long. Google was mostly already there so I can’t really take credit for it. Now I am busy raising the bar further.

uTest: Top tester Glory Leung is curious: What are your views on SCRUM testing in general? Are people doing it properly? What is the ideal situation?

JW: Scrum is just a name. I don’t like names, they feel too confining and people have their own ideas of what they mean. I took a lot of flak for using the name ‘exploratory testing’ for my book. People love to confine you to how they view a specific named idea or technique. Flexibility is required.

Continue Reading

And Action! uTest Hits The Road: Boston to NYC to San Diego & Back

Whoever said “things slow down in the summertime” does not know uTest! Over the next couple of weeks, uTest is hitting the road again — trekking from the E2.0 Conference in Boston to the Venture Summit in NYC to Red Herring in San Diego and back!

Today @ 3:30pm: Matt Johnston will speak at the prolific Enterprise 2.0 Conference in Boston on the topic of — “You Say Social Media, I Say Community: Does It Matter?”

Panelists and community experts Eran Barak (Thomson Reuters), Michael Petillo (W.L. Gore), Megan Murray (Booz Allen Hamilton), Matt from uTest (of course!) and Moderator Rachel Happe (The Community Roundtable) will discuss the important distinctions between ‘social media’ and ‘community,’ and how they approach the challenges of utilizing social media and online communities in different types of organizations.

Thurs, June 17th @ 10:00am: Doron Reuveni will present at the exclusive 2010 New York Venture Summit. He will also be accepting — for the second year in a row — a “Top Innovator Award” by youngStartup Ventures, which recognizes cutting-edge private companies driving the future of innovation, especially in the tech sector. This event will be held at The Hotel Penn in New York City.

Stay tuned for next week’s adventures to San Diego and more from Boston!

uTest Opens West Coast Office

Today we’re excited to announce the grand opening of uTest’s west coast office in Sunnyvale, CA (a.k.a. Silicon Valley) — home to our fastest growing territory!

In addition to several uTest employees already based in the Bay Area, the new office will put some much needed “feet-on-the-street” to support our large roster of customers and partners on the west coast.

In 2010, uTest will add more project managers, sales and marketing employees to provide a personal presence and an even higher level of service to our customers.

Keep an eye on our careers page in the coming months for open positions (if you’re in the Boston area, we have open positions now)!

The Sunnyvale office joins uTest’s global teams, including uTest HQ near Boston and operations in London and Tel Aviv.

Version 3.0.2 – Tester Snapshots, Favorite Testers and More

Late last night, our engineering crew released version 3.0.2 of the uTest platform. This update contains several larger features and a bunch of small ones — all based on insights we gathered from our customers and testers. Here are a few of the highlights:

Favorite Tester
As customers run more and more test cycles, they often identify favorite testers they want to invite back again and again. These recurring relationships are good for customers and the testers alike. In the past, customers had to keep track of their favored testers by hand. But with our new favorite tester feature we’re making it easy for customers to remember the testers they like best, and to invite them back over and over.

Customers can now more easily see which testers participated in their past test cycles as well as how they performed. By clicking a yellow star icon next to a tester’s name, a customer can mark that particular tester as one of their favorites. When that customer runs a new test cycle, their favorite testers are placed at the top of the list, making inviting them back a snap.

Tester Snapshot
Wouldn’t it be nice for testers to have a place to show off their ratings along with an overview of their location, testing environment, and maybe a photo of themselves? With this launch we’ve made that idea a reality with our new tester snapshot feature.

Testers and customers can click on any tester’s name in the platform and see the public tester snapshot for that tester (location, testing environments and a photo). Testers who are rated or who have been marked as a favorite tester will also have some snazzy new badges that show off their uTest skills. This information can be viewed by other testers or customers.

If a tester clicks on their own name in the platform, they can also see their private tester snapshot (including past performance and earnings history). Just click “Statistics” and your private tester snapshot will display your past test cycles, your bug approval percentages and more. This information cannot be viewed by other testers or customers.

All the Rest
Other improvements include:

  • Improved workflow for submitting and approving test cases
  • More intelligent tester notification emails
  • Numerous performance tweaks
  • Dozens of bug fixes and other minor enhancements

Have a great idea for our future product releases? uTest community members can join our tester forums and check out our Platform Feedback section. Customers can contact their project manager directly or drop us a line.

1,000 Twitter Followers & 500 Facebook Likes Later…

Today I read a story about a woman who, while following Google Maps’ directions, was run over and is now suing Google for damages. So, my question to you is: Who are you following — and why?

At uTest, we’re just scratching the surface of  what’s possible in our “social media” efforts, but we’re excited that this past weekend we passed 1,000 Twitter Followers and 500 Facebook Likes!

So this post is simply to ask YOU — our terrific community, friends and readers — a few questions about what makes a company worth following or more interesting to you in the realm of social media:

  1. What type of content do you like most? Is it breaking news; thoughts from industry gurus; inside info from the company; jokes and funny stories; special promotions; or other?
  2. What makes you want to follow or “like” a company — particularly a B2B brand?
  3. Are there other B2B or SaaS companies who are doing cool stuff and making all the right moves?  Give ‘em some love and tell the world about them in the comments!

We would love to hear your thoughts, ideas and insights around this topic. And if you have a moment, please follow us (we promise we won’t run you over!).

To Crowdsourcing Friends, Foes & Fanatics: Just How Loyal Is Your Community?

Depending upon who you ask, crowdsourcing is either evil, revolutionary, or a next gen of internships.

But one thing that ALL crowdsourcing companies like to preach is how loyal and trustworthy and professional their community is. I know because I’ve read it in 100 different sites. Hell, I’ve written it a 100 different times here at uTest. So why do crowdsourcing companies insist upon telling the world how loyal and earnest their community is?  Maybe it’s to assuage the fears of prospective customers about entrusting their logo design, app development, content production or marketing to a community of strangers. Maybe it’s because if marketers say it enough times, we hope it’ll come true.

The more pessimistic view is that people — cloaked in the anonymity of the web — often act in greedy, selfish, mean-spirited ways (this perspective didn’t make it into the crowdsourcing brochure, by the way). Such dark behavior is well-documented and takes the form of flame wars on message boards, bullying via social media and online fraud.

So which is it — are people good-natured and honest?  Or are they money-hungry malcontents who will do anything to get ahead, as long as they don’t get caught?  Obviously it depends on the people, but I learned the truth about our community this week — and it was a lesson we learned the hard way.

A little background:  At uTest, we pay our testers twice per month via PayPal or Payoneer. And at this point in our growth, each pay cycle involves a non-trivial amount of cash — pretty deep into the five-figure range.  Now, it’s not easy or flattering to admit this, but in our most recent pay cycle, we experienced a glitch that caused us to pay our testers twice. That’s right folks, it was double payday here at uTest!

Continue Reading

I Love The Smell Of Bug Battles In The Morning…

No bugs were harmed in the planning of this Bug Battle… but they will be — starting Friday at noon ET.  That’s right uTesters — it’s time for our 2nd quarter Bug Battle competition!

And this time, we’ll be checking out the top location-based check-in services: Foursquare vs. Gowalla vs. Brightkite.  We’ll be letting our community put these innovators to the test (their public-facing websites, their check-in services, their mobile apps and their integrations with Facebook and Twitter).

And after the testing phase is complete on Monday, May 24th, we’ll be sending a survey to all participating testers to compare the usability and feature set of these three leaders.

What’s in it for testers?  Well, bragging rights, sure.  But we’ll also be doling out nearly $4,000 in prize money to the testers who report the most severe/interesting bugs and provide the most insightful survey feedback.

So it’s time to do what you do best, uTesters!  Log into your uTest account and scour these apps for quality defects and report them in a clear, concise way.  And if you do it better than your peers, you could be named the Q2 Bug Battle winner and earn some big prize money for your time.

uTest Up For 3 Stevie Awards, Including “Most Innovative Company”

Last year, we were psyched to take home the “Best New Company of the Year” for 2009 at the American Business Awards (a.k.a. The Stevies). This year, we’re even more fired up to let you know that the members of the Stevie Awards’ Board have named uTest a finalist in three top categories:

1. Most Innovative Company Of The Year
2. Best Blog in the Computer Services category (for this blog) and
3. B2B Communications Campaign Of The Year
(for the Bug Battle!)

Although we’ve made the shortlist, we definitely have our work cut out for us to bring in these big wins. Take the Comm Campaign category for example — we’re up against huge software leaders, including EMC and Novell. See the complete list of finalists here!

The American Business Awards are the U.S.’s premier business awards program. More than 2,700 entries in virtually every industry were submitted for consideration this year. The winners will be announced on Monday, June 21 at the Marriott Marquis Hotel in Times Square/NYC. More than 600 execs from around the globe are expected to attend.

Let us know if you’ll be in NYC that week. We’d love to meet up and fill you in on the latest!

Q2 Bug Battle: Foursquare vs. Gowalla vs. Brightkite

Where are you going, where have you been? Yes it’s the title of a creepy short story, but it also happens to characterize the  fastest-growing segment of consumer applications.

Location-based apps like Foursquare, Gowalla and Brightkite – apps that let you check in, find hot spots and get rewarded with prizes & accolades – are attracting thousands of new users every day. Just how popular have they become? They’re sooo popular, even uTest CEO and co-founder Doron Reuveni has attained ‘Mayor’ status (see if you can guess where).

Why am I telling you all this? Because these  geo-based apps are the subject of our latest uTest Bug Battle. We figured that since Foursquare, Gowalla and Brightkite are all global in scope, that they would provide our global tester community with a challenge unlike any of our previous bug-hunting competitions.

The Bug Battle gets started this Friday, May 14 at Noon (ET). Testers will be given ten days to search these applications (both web and mobile) for the most compelling bugs, and to report them through our online platform. We’ll be dishing out nearly $4,000 in prize money for Top Tester, Best Bug, Best Survey Feedback, Best Mobile Bug and a bunch of other categories.

More details on this Bug Battle – as well the rules, prizes and deadlines –can be found in this uTest Forums thread. Of course, we’ll be providing frequent updates via our blog, forums and newsletters.

Special note: This competition is available ONLY to members of the uTest community. Not a member yet? Find out more about the benefits of joining uTest.