If you were going to invest more money in testing, where would you place those bets? Testing early in the cycle? Automation? Manual testing? Better requirements and planning? Better documentation? Torture devices for your devs?

In our latest uTest webinar, More Bang For Your Testing Buck, best-selling author James Whittaker will take a critical look at how to maximize your investments in testing, including an examination of the tools, best practices and methodologies that can help make testing a “happier place.”

The webinar is schedule for Friday, December 10 at 11:00 AM (EST).

Register now at Go-To Meeting >>>>

As the former Test Architect at Microsoft and the current Director of Testing at Google, James is one of the most accomplished professionals in the testing business. Long-time readers of our blog will know that he is also no stranger to the uTest community. Here are a few of the uTest projects he’s participated in:

Read more…

As uTest’s October Tour finally winds down, we wanted to share a few more pics with you from abroad. As you know, uTest exec John Montgomery was selected to attend the invite-only Google Test Automation Conference (GTAC) in Hyderabad this week (uTest’s second year in a row!). A few little Twitter birds (Pradeep Soundararajan and Joel Hynoski) also announced that uTest was mentioned several times from the stage during talks of crowdsourced testing and Mozilla’s crowd strategy. A fantastic event overall! Click here for all Day One and Day Two pics.

Here are a few of our faves (click to expand)…

GTAC begins!

John networks with GTAC attendees

Pradeep Soundararajan presenting

GTAC crew huddles together for a quick pic

Can you find John? Click for hint.

Doc James Whittaker presenting

Check out the spread!

Delicious authentic Indian food

October is bursting at the seams in terms of uTest speaking engagements! The exec team will be presenting at leading events all over the world this month — from San Fran to London to Hyderabad and back.

SAN FRANCISCO 10/4-10/7
We start in San Francisco next week at CrowdConf, the world’s first conference dedicated to the world of crowdsourcing and the future of work. CrowdConf will be held at the St. Regis Hotel on October 4th.

Crowdsourcing veterans CEO of oDesk, Gary Swart, and CEO of Elance, Fabio Rosati, will join Doron on a panel to discuss the ways crowdsourcing models are used to maximize a company’s potential and manage costs. More details here!

By the way, we will also be at CTIA in San Fran October 5th – October 7th at both the Mobile Web & Apps Forum (10/5) and the iPad & Tablet Publishing/Entertainment Apps (10/6) event. CTIA will be held at Moscone Center West. Let us know if you’ll be there!

LONDON 10/13 & 10/19
Next stop – the UK! On October 13th, Co-Founder of uTest Roy Solomon will be speaking at TCL’s Star Testing event in London. Other speakers include our friends James Whittaker (two-time Testing The Limits veteran!) and Tom Lounibos, CEO of SOASTA. This event will be held at One Alfred Place.

On October 19th, uTest exec Matt Johnston will be among the outstanding line-up of more than 40 speakers at Mobile App World, which includes Google, Microsoft, Ericsson, Orange Global and the BBC, who will be discussing the future of mobile apps.

HYDERABAD 10/28
Next stop – India! uTest was selected to attend the invite-only Google Test Automation Conference (GTAC) in Hyderabad on October 28 for the second year in a row! uTest exec John Montgomery will be there.

DEARBORN 10/28
Last but not least, Matt will be speaking at TiE CON 2010 in Dearborn, Michigan on October 28 at 1pm in the Crowdsourcing track.

And that’s it! (Where’s an exhausted emoticon when you need one?) If you want to meet up with us at any of these events, please shoot us a note. We’d love to share a coffee break with you.

In the second part of our Testing the Limits interview with James Whittaker, we tackle Google vs. Microsoft; dogs vs. cats; why SCRUM is just a name; his advice for college graduates; bad habits of exploratory testing and more. If you missed Part I, you can find it here.

If you want to read more of James’ work, bookmarking the Google Testing Blog would be a good place to start. You can also read his 2009 book Exploratory Software Testing or check out some of his uTest eBooks and webinars.

uTest: The Microsoft vs. Google battle continues to play out very publicly in the media. Just last week, Computerworld wrote this story: “Microsoft: No Matter What Google Says, Windows Is Secure.” Having been at both companies, we think you have a unique perspective on this one. Any thoughts?

JW: Let me say right away that I enjoyed my time at Microsoft and admire the company and the smart people who work there. As a resident of Seattle, it is in my best interest for Microsoft to prosper! But the two companies are vastly different regarding the way their talent is managed and their products are built. Google is an engineering-centric company where innovation comes from individuals who are empowered to do whatever they need to get ideas into production. Much has been made of Google’s game-theory approach to managing people where rewards are given quickly for impactful behavior. It works. Morale is high and people work very hard and take quality very seriously.

Does this mean we produce more secure or more reliable products? We try hard to do so; Microsoft tries equally hard. I think we have the advantage of less legacy and a more modern and reliable platform (the Web as opposed to client operating systems) to work from. But the secret sauce at both companies is the same: hard work and due diligence.

uTest: You shared with us (as the pioneer of Testing the Limits posts) that your first assignment at Google was “To raise the level of testing precision and diligence.” So, how did it go?

JW: It didn’t take long. Google was mostly already there so I can’t really take credit for it. Now I am busy raising the bar further.

uTest: Top tester Glory Leung is curious: What are your views on SCRUM testing in general? Are people doing it properly? What is the ideal situation?

JW: Scrum is just a name. I don’t like names, they feel too confining and people have their own ideas of what they mean. I took a lot of flak for using the name ‘exploratory testing’ for my book. People love to confine you to how they view a specific named idea or technique. Flexibility is required.

Read more…

It was one year ago (June ’09) that James Whittaker helped us christen our ‘Testing The Limits’ interview series by being our first guest. And for much of the year, he held the distinction of generating the most page views… and then some guy named Patrick Copeland came along and took the lead a few months back.

Well, in honor of our one-year anniversary, James has accepted our invite to be our first-ever return guest – and this marks the start of a new trend. In our 2nd year of Testing the Limits, we’re going to be revisiting some of the past legends and leaders to see what’s changed since they last spoke with us. Of course, we’ll also be blending in some voices we haven’t heard from yet (we’re looking at you, Cem Kaner and Elizabeth Hendrickson) so stay tuned!

In this interview, James discusses his present role at Google; the emergence of Web Test Framework (aka WTF); the next decade of testing innovations; cloud computing and much more. When you’re done with this one, go read Part II.

uTest: A year ago, the big news was about your move from Microsoft to Google. Now that you’re no longer a Noogler, how has this year changed your perspective on testing and the testing industry? What has surprised you most?  Can you share any favorite stories?

JW: Four years ago I made the decision to leave the comfy confines of academe and consultancy and do something more real. It seems there is a steady supply of ex-industry folks going into consulting and not much of a flow the other way. I thought it would challenge me more than anything else I could do. Unfortunately, Microsoft just wasn’t the place to pull that off, ship schedules in the client-server domain simply didn’t allow a fast enough pace to suit me. I’ve been part of more software development in the past year at Google than I had my entire time at Microsoft and my consulting career combined. Things I didn’t think possible like shipping a product from concept to production in a matter of weeks, doing software development in a way that makes testing mostly invisible and creating completely new uses for test techniques that I had dismissed as amateur earlier in my career (e.g., record and playback) have not only surprised me but also now make my days a lot more interesting.

Another perspective that has changed is my appreciation of automated testing has grown. I’ve written extensively about manual testing and the importance of having a brain in-the-loop and I haven’t given it enough credit to automation in the past. Automation is really important and I think the detractors to it, simply don’t know how to do it right or simply don’t have enough experience with it. At the same time my appreciation for manual testing has grown as well, but I no longer advocate doing it without a lot of automated assistance. I’ll explain more about that later.

uTest: In the spirit of “WTF”, can you tell us more about the new, appropriately named, Web Test Framework and the unique control that Chrome and Chrome OS will offer web apps, browsers and the operating systems they are running on?

JW: I work with a developer who believes that WTF (the real meaning of the acronym) is the only appropriate response to a tester who creates yet another test framework. I have to admit, it is a response I often employ as well. Does the world really need another test framework? What the —-?

Well the world needs this one.

Read more…

Today is Memorial Day here in the states. It’s a day when we honor those who serve our country — both past and present. At the same time, I’m mindful that our readers comes from a global community and this blog isn’t about politics or ideologies. So I’ll set aside my personal beliefs and offer up a short post about remembering those who preceded us professionally – yesterday’s pioneers and today’s mentors.

At some point along the way, each of us has had someone (or if we’re lucky, multiple people) who have guided and shaped us in the professional realm.

If you’re a tester, it may be the teachings of thought leaders like James Whitaker or James Bach, or possibly an organization like STP or the Weekend Testers. If you’re a developer, it might be an individual like Joel Spolsky or the fine folks at Slashdot that helped spark your interest. But for many of us startup junkies, it was a professor, a boss, an investor, or some other form of professional mentor. These people usually aren’t published or famous. Nevertheless, they helped make us who we are as professionals.

So take a moment and think about who has guided you in your career, either directly or indirectly. Then think about how you can do more to contribute to someone else’s journey.

For my part, I’m going to be thinking a lot more about this with my team and with our community at large. And from a personal perspective, sincere thanks to Bob, Greg and Paul for helping me to find solid footing at different points along my journey.

As a native English speaker, the pronunciation of Iceland’s volcano has eluded me.  But I knew Icelandic was a tough language from when I visited there a year ago (see my post: Six Testing Lessons From Iceland).  What I also know is that travel in Europe has turned into a mess, and that travelers from around the world have suffered from the wrath of Eyjafjwhatever (including uTest’s very own product manager who is currently stuck in the United Kingdom).

Of course, getting stuck on a vacation to a foreign country sounds awful (I have terrible nightmares of one day getting stuck in Tahiti), but there are some valuable lessons we can learn from all this.

1. Expect the unexpected.
In my testing lessons from Iceland, I invented a few Icelandic testing tours in the spirit of James Whitaker.  One of them was the Heimaey tour:

Heimaey – The Icelandic island of Heimaey is known for two things: fishing and its little problem with volcanoes.  In 1973, the entire island was evacuated after massive cracks formed in the ground spewing lava and ash everywhere – eventually forming the volcano Eldfell.  Take a tour of your users’ worst nightmare problems and make sure everything works correctly.

Read more…

Last week, we found out that our humble little Software Testing Blog won the Hive Award at SXSW as the top business software blog (here’s the slideshow and the PDF report). We’re honored to make this prestigious list, along with brands we love such as HowStuffWorks, Nokia, Nike, HBO and About.com.

Part of the reason this blog has been so successful in the past year is how infrequently we talk about ourselves (ugh, boring). Well, I’m allowing myself to break that rule briefly so I can thank the people who have made our blog what it is today.

• Our in-house team (Stanton, Mike, Jenny and Peter) for their tireless efforts and talented writing about everything from mobile apps to social media to software testing to crowdsourcing trends.
• Our guest bloggers from the uTest community who have written passionately about everything from mobile testing to QA in agile environments to the evolving roles of testers.
• Our Testing The Limits guests (including James Whittaker, Matt Heusser, James Bach, Michael Bolton and Jon Bach) who have not only tolerated our wide range of questions — from the insightful to the inane — but joined in with good humor, wit, eloquence and intellect.

I’ll end this little Oscar speech before the orchestra starts playing me off stage. Suffice it to say, we love writing for you; we’ll keep scouring every corner of the world (virtual and physical) for fresh topics and angles about anything related to software; and we’ll keep reminding ourselves why we’ve had this success: we write stuff that you seem to enjoy reading. We now return you to your regularly scheduled programming.

Well, today he’s sticking close to home in Boston. Tomorrow he’ll land in London… and before the week is out, he’ll hit Tel Aviv.

Doron starts Wednesday morning off (after his usual 10-mile run, of course!) in London with some tea and networking with friend and colleague, James Whittaker and UK partner, TCL.

Then he’s off to QCon London, an excellent conference for the enterprise software community. On Friday, 3/12 @ 2pm, he’ll be presenting at QCon re: The Mobile App Quality Challenge & How Crowdsourcing Can Help.

Doron is one of five software testing leaders chosen to present in the “How Do You Test That?” track. This track explores unique solutions created to address situations in which automated testing does not suffice.

And on the last leg of his marathon journey, Doron will present at Garage Geeks in Israel on Monday, 3/15 @ 8pm. There, Doron will be taking a deep dive into the topic of Crowdsourcing, and how smart recruiting, training and incentives can turn an unstructured, loosely assembled mob into a unified, professional community.

So, where in the world is Doron this week?  Catch him if you can!

In this month’s Testing the Limits interview, we’ll put Patrick Copeland on the hot seat. Patrick is the Senior Engineering Director for a promising young upstart named Google (we’re not familiar with them ourselves, but we’ve heard good things) where he oversees a global team of about 800 engineers. But this isn’t his first rodeo –  prior to Google, Patrick spent a decade at Microsoft, where he specialized in all things related to software engineering.

So what do you ask someone who’s probably forgotten more about software than we’ll ever know? Well, in this installment, we’re going to get his views on catering to a global base of users; his criteria for evaluating testers based on their “tester DNA”; the recent addition of our good friend James Whittaker; the challenges of launching new products like the Nexus One, as well as other tidbits from inside the GooglePlex. Stay tuned for Parts II and III in the days ahead.

uTest: What are some of the challenges that come with having a global base of customers and users? Are certain products noticeably more popular in some areas rather than others? And how does this affect your future planning?

PC: Yes, of course some products and features do better than others. Our approach is to do lots of experimentation and to release and iterate. We push bits to customers early and often, and then we listen and watch usage. Customers help us by “voting with their feet.” Popular features and products are improved, and poorly performing products are deprecated. With a big focus on innovation, we also need to “fail fast” and customer feedback helps us make those decisions.

Not surprising, our global customers have different demands of our products. We want products to “feel local” and we need to support features that may be unique to specific markets. For instance, in Indic based languages using a standard keyboard is difficult, so we develop strategies like virtual keyboards or category browsing for search. As we specialize our products for certain markets, it introduces more challenges for testing (eg. requiring special cultural knowledge). When we can’t find internal talent, community-based testing is an interesting solution to this challenge.

We base staffing and planning decisions on several criteria:

• Strategic: Maybe a new feature, but in a market with existing competition (like Android).
• Financial: Obviously Ads and Search, but we have several emerging businesses that are also getting important.
• Customer usage: For example, popular high-traffic applications like GMail.
• Legal or Compliance: Certain areas need to be prioritized high for legal reasons. For example, SOX compliance for CheckOut.
• Ability to Impact: We look at our capability and decide if investing testers in an area would have a significant impact.

uTest: A few years back, you were the keynote speaker at GTAC, where you said something to the effect that “the longer I’ve been in the business, the less I know about it.” How important is it for testers and developers (and those who manage them) to maintain this student-for-life mindset?

PC: Very. When I hire people I look for folks with a “testing DNA.” These are people who are great computer scientists at their core, but also are very curious, love software, and are passionate about test engineering. People who have those characteristics tend to pursue challenges and continue to learn.

Read more…