Tag Archives | james whittaker

uTest On Tour – San Fran to London to Hyderabad and Back

October is bursting at the seams in terms of uTest speaking engagements! The exec team will be presenting at leading events all over the world this month — from San Fran to London to Hyderabad and back.

We start in San Francisco next week at CrowdConf, the world’s first conference dedicated to the world of crowdsourcing and the future of work. CrowdConf will be held at the St. Regis Hotel on October 4th.

Crowdsourcing veterans CEO of oDesk, Gary Swart, and CEO of Elance, Fabio Rosati, will join Doron on a panel to discuss the ways crowdsourcing models are used to maximize a company’s potential and manage costs. More details here!

By the way, we will also be at CTIA in San Fran October 5th – October 7th at both the Mobile Web & Apps Forum (10/5) and the iPad & Tablet Publishing/Entertainment Apps (10/6) event. CTIA will be held at Moscone Center West. Let us know if you’ll be there!

LONDON 10/13 & 10/19
Next stop – the UK! On October 13th, Co-Founder of uTest Roy Solomon will be speaking at TCL’s Star Testing event in London. Other speakers include our friends James Whittaker (two-time Testing The Limits veteran!) and Tom Lounibos, CEO of SOASTA. This event will be held at One Alfred Place.

On October 19th, uTest exec Matt Johnston will be among the outstanding line-up of more than 40 speakers at Mobile App World, which includes Google, Microsoft, Ericsson, Orange Global and the BBC, who will be discussing the future of mobile apps.

Next stop – India! uTest was selected to attend the invite-only Google Test Automation Conference (GTAC) in Hyderabad on October 28 for the second year in a row! uTest exec John Montgomery will be there.

Last but not least, Matt will be speaking at TiE CON 2010 in Dearborn, Michigan on October 28 at 1pm in the Crowdsourcing track.

And that’s it! (Where’s an exhausted emoticon when you need one?) If you want to meet up with us at any of these events, please shoot us a note. We’d love to share a coffee break with you.

Continue Reading

Testing the Limits with James Whittaker – Part II

In the second part of our Testing the Limits interview with James Whittaker, we tackle Google vs. Microsoft; dogs vs. cats; why SCRUM is just a name; his advice for college graduates; bad habits of exploratory testing and more. If you missed Part I, you can find it here.

If you want to read more of James’ work, bookmarking the Google Testing Blog would be a good place to start. You can also read his 2009 book Exploratory Software Testing or check out some of his uTest eBooks and webinars.

uTest: The Microsoft vs. Google battle continues to play out very publicly in the media. Just last week, Computerworld wrote this story: “Microsoft: No Matter What Google Says, Windows Is Secure.” Having been at both companies, we think you have a unique perspective on this one. Any thoughts?

JW: Let me say right away that I enjoyed my time at Microsoft and admire the company and the smart people who work there. As a resident of Seattle, it is in my best interest for Microsoft to prosper! But the two companies are vastly different regarding the way their talent is managed and their products are built. Google is an engineering-centric company where innovation comes from individuals who are empowered to do whatever they need to get ideas into production. Much has been made of Google’s game-theory approach to managing people where rewards are given quickly for impactful behavior. It works. Morale is high and people work very hard and take quality very seriously.

Does this mean we produce more secure or more reliable products? We try hard to do so; Microsoft tries equally hard. I think we have the advantage of less legacy and a more modern and reliable platform (the Web as opposed to client operating systems) to work from. But the secret sauce at both companies is the same: hard work and due diligence.

uTest: You shared with us (as the pioneer of Testing the Limits posts) that your first assignment at Google was “To raise the level of testing precision and diligence.” So, how did it go?

JW: It didn’t take long. Google was mostly already there so I can’t really take credit for it. Now I am busy raising the bar further.

uTest: Top tester Glory Leung is curious: What are your views on SCRUM testing in general? Are people doing it properly? What is the ideal situation?

JW: Scrum is just a name. I don’t like names, they feel too confining and people have their own ideas of what they mean. I took a lot of flak for using the name ‘exploratory testing’ for my book. People love to confine you to how they view a specific named idea or technique. Flexibility is required.

Continue Reading →

Continue Reading

Testing the Limits with James Whittaker – Part I

It was one year ago (June ’09) that James Whittaker helped us christen our ‘Testing The Limits’ interview series by being our first guest. And for much of the year, he held the distinction of generating the most page views… and then some guy named Patrick Copeland came along and took the lead a few months back.

Well, in honor of our one-year anniversary, James has accepted our invite to be our first-ever return guest – and this marks the start of a new trend. In our 2nd year of Testing the Limits, we’re going to be revisiting some of the past legends and leaders to see what’s changed since they last spoke with us. Of course, we’ll also be blending in some voices we haven’t heard from yet (we’re looking at you, Cem Kaner and Elizabeth Hendrickson) so stay tuned!

In this interview, James discusses his present role at Google; the emergence of Web Test Framework (aka WTF); the next decade of testing innovations; cloud computing and much more. When you’re done with this one, go read Part II.

uTest: A year ago, the big news was about your move from Microsoft to Google. Now that you’re no longer a Noogler, how has this year changed your perspective on testing and the testing industry? What has surprised you most?  Can you share any favorite stories?

JW: Four years ago I made the decision to leave the comfy confines of academe and consultancy and do something more real. It seems there is a steady supply of ex-industry folks going into consulting and not much of a flow the other way. I thought it would challenge me more than anything else I could do. Unfortunately, Microsoft just wasn’t the place to pull that off, ship schedules in the client-server domain simply didn’t allow a fast enough pace to suit me. I’ve been part of more software development in the past year at Google than I had my entire time at Microsoft and my consulting career combined. Things I didn’t think possible like shipping a product from concept to production in a matter of weeks, doing software development in a way that makes testing mostly invisible and creating completely new uses for test techniques that I had dismissed as amateur earlier in my career (e.g., record and playback) have not only surprised me but also now make my days a lot more interesting.

Another perspective that has changed is my appreciation of automated testing has grown. I’ve written extensively about manual testing and the importance of having a brain in-the-loop and I haven’t given it enough credit to automation in the past. Automation is really important and I think the detractors to it, simply don’t know how to do it right or simply don’t have enough experience with it. At the same time my appreciation for manual testing has grown as well, but I no longer advocate doing it without a lot of automated assistance. I’ll explain more about that later.

uTest: In the spirit of “WTF”, can you tell us more about the new, appropriately named, Web Test Framework and the unique control that Chrome and Chrome OS will offer web apps, browsers and the operating systems they are running on?

JW: I work with a developer who believes that WTF (the real meaning of the acronym) is the only appropriate response to a tester who creates yet another test framework. I have to admit, it is a response I often employ as well. Does the world really need another test framework? What the —-?

Well the world needs this one.

Continue Reading →

Continue Reading

Remembering Those Who Helped Us Along The Way

Today is Memorial Day here in the states. It’s a day when we honor those who serve our country — both past and present. At the same time, I’m mindful that our readers comes from a global community and this blog isn’t about politics or ideologies. So I’ll set aside my personal beliefs and offer up a short post about remembering those who preceded us professionally — yesterday’s pioneers and today’s mentors.

At some point along the way, each of us has had someone (or if we’re lucky, multiple people) who have guided and shaped us in the professional realm.

If you’re a tester, it may be the teachings of thought leaders like James Whitaker or James Bach, or possibly an organization like STP or the Weekend Testers. If you’re a developer, it might be an individual like Joel Spolsky or the fine folks at Slashdot that helped spark your interest. But for many of us startup junkies, it was a professor, a boss, an investor, or some other form of professional mentor. These people usually aren’t published or famous. Nevertheless, they helped make us who we are as professionals.

So take a moment and think about who has guided you in your career, either directly or indirectly. Then think about how you can do more to contribute to someone else’s journey.

For my part, I’m going to be thinking a lot more about this with my team and with our community at large. And from a personal perspective, sincere thanks to Bob, Greg and Paul for helping me to find solid footing at different points along my journey.

Continue Reading


As a native English speaker, the pronunciation of Iceland’s volcano has eluded me.  But I knew Icelandic was a tough language from when I visited there a year ago (see my post: Six Testing Lessons From Iceland).  What I also know is that travel in Europe has turned into a mess, and that travelers from around the world have suffered from the wrath of Eyjafjwhatever (including uTest’s very own product manager who is currently stuck in the United Kingdom).

Of course, getting stuck on a vacation to a foreign country sounds awful (I have terrible nightmares of one day getting stuck in Tahiti), but there are some valuable lessons we can learn from all this.

1. Expect the unexpected.
In my testing lessons from Iceland, I invented a few Icelandic testing tours in the spirit of James Whitaker.  One of them was the Heimaey tour:

Heimaey – The Icelandic island of Heimaey is known for two things: fishing and its little problem with volcanoes.  In 1973, the entire island was evacuated after massive cracks formed in the ground spewing lava and ash everywhere – eventually forming the volcano Eldfell.  Take a tour of your users’ worst nightmare problems and make sure everything works correctly.

Continue Reading →

Continue Reading

uTest Blog Abuzz With Hive Award Win @ SXSW

Last week, we found out that our humble little Software Testing Blog won the Hive Award at SXSW as the top business software blog (here’s the slideshow and the PDF report). We’re honored to make this prestigious list, along with brands we love such as HowStuffWorks, Nokia, Nike, HBO and About.com.

Part of the reason this blog has been so successful in the past year is how infrequently we talk about ourselves (ugh, boring). Well, I’m allowing myself to break that rule briefly so I can thank the people who have made our blog what it is today.

  • Our in-house team (Stanton, Mike, Jenny and Peter) for their tireless efforts and talented writing about everything from mobile apps to social media to software testing to crowdsourcing trends.
  • Our guest bloggers from the uTest community who have written passionately about everything from mobile testing to QA in agile environments to the evolving roles of testers.
  • Our Testing The Limits guests (including James Whittaker, Matt Heusser, James Bach, Michael Bolton and Jon Bach) who have not only tolerated our wide range of questions — from the insightful to the inane — but joined in with good humor, wit, eloquence and intellect.

I’ll end this little Oscar speech before the orchestra starts playing me off stage. Suffice it to say, we love writing for you; we’ll keep scouring every corner of the world (virtual and physical) for fresh topics and angles about anything related to software; and we’ll keep reminding ourselves why we’ve had this success: we write stuff that you seem to enjoy reading. We now return you to your regularly scheduled programming.

Continue Reading

Where In The World Is Doron Reuveni?

Well, today he’s sticking close to home in Boston. Tomorrow he’ll land in London… and before the week is out, he’ll hit Tel Aviv.

Doron starts Wednesday morning off (after his usual 10-mile run, of course!) in London with some tea and networking with friend and colleague, James Whittaker and UK partner, TCL.

Then he’s off to QCon London, an excellent conference for the enterprise software community. On Friday, 3/12 @ 2pm, he’ll be presenting at QCon re: The Mobile App Quality Challenge & How Crowdsourcing Can Help.

Doron is one of five software testing leaders chosen to present in the “How Do You Test That?” track. This track explores unique solutions created to address situations in which automated testing does not suffice.

And on the last leg of his marathon journey, Doron will present at Garage Geeks in Israel on Monday, 3/15 @ 8pm. There, Doron will be taking a deep dive into the topic of Crowdsourcing, and how smart recruiting, training and incentives can turn an unstructured, loosely assembled mob into a unified, professional community.

So, where in the world is Doron this week?  Catch him if you can!

Continue Reading

Testing the Limits with Google’s Patrick Copeland – Part I

In this month’s Testing the Limits interview, we’ll put Patrick Copeland on the hot seat. Patrick is the Senior Engineering Director for a promising young upstart named Google (we’re not familiar with them ourselves, but we’ve heard good things) where he oversees a global team of about 800 engineers. But this isn’t his first rodeo –  prior to Google, Patrick spent a decade at Microsoft, where he specialized in all things related to software engineering.

So what do you ask someone who’s probably forgotten more about software than we’ll ever know? Well, in this installment, we’re going to get his views on catering to a global base of users; his criteria for evaluating testers based on their “tester DNA”; the recent addition of our good friend James Whittaker; the challenges of launching new products like the Nexus One, as well as other tidbits from inside the GooglePlex. Stay tuned for Parts II and III in the days ahead.

uTest: What are some of the challenges that come with having a global base of customers and users? Are certain products noticeably more popular in some areas rather than others? And how does this affect your future planning?

PC: Yes, of course some products and features do better than others. Our approach is to do lots of experimentation and to release and iterate. We push bits to customers early and often, and then we listen and watch usage. Customers help us by “voting with their feet.” Popular features and products are improved, and poorly performing products are deprecated. With a big focus on innovation, we also need to “fail fast” and customer feedback helps us make those decisions.

Not surprising, our global customers have different demands of our products. We want products to “feel local” and we need to support features that may be unique to specific markets. For instance, in Indic based languages using a standard keyboard is difficult, so we develop strategies like virtual keyboards or category browsing for search. As we specialize our products for certain markets, it introduces more challenges for testing (eg. requiring special cultural knowledge). When we can’t find internal talent, community-based testing is an interesting solution to this challenge.

We base staffing and planning decisions on several criteria:

  • Strategic: Maybe a new feature, but in a market with existing competition (like Android).
  • Financial: Obviously Ads and Search, but we have several emerging businesses that are also getting important.
  • Customer usage: For example, popular high-traffic applications like GMail.
  • Legal or Compliance: Certain areas need to be prioritized high for legal reasons. For example, SOX compliance for CheckOut.
  • Ability to Impact: We look at our capability and decide if investing testers in an area would have a significant impact.

uTest: A few years back, you were the keynote speaker at GTAC, where you said something to the effect that “the longer I’ve been in the business, the less I know about it.” How important is it for testers and developers (and those who manage them) to maintain this student-for-life mindset?

PC: Very. When I hire people I look for folks with a “testing DNA.” These are people who are great computer scientists at their core, but also are very curious, love software, and are passionate about test engineering. People who have those characteristics tend to pursue challenges and continue to learn.

Continue Reading →

Continue Reading

Testing The Limits — 2009’s Top Posts

Testing The LimitsAfter we re-launched our brand in May, we decided that the uTest blog needed to be more than just uTest employees talking about uTest events, uTest awards and the uTest community (see how repetitive that gets?).

Writing witty, thought-provoking content is really hard.  And we’re pretty lazy, but fortunately we know some extremely smart & funny people.  So we invented the Testing The Limits series, in which we interview leaders from the worlds of testing, software, entrepreneurship and crowdsourcing.

We’re immensely grateful to these talented, busy people, and we have much more planned for the Testing The Limits series in 2010.  But before we flip the calendar, these posts from this year are worth another look:

June: James Whittaker — Author, Professor and Testing Evangelist at Google

July: Rosie Sherry — Founder of the UK-based Software Testing Club

August: Andrew Muns — President of Software Test & Performance

September: Jack Margo — SVP of Internet Operations of Developer Shed

October: Jon Winsor — Author, Crowdsourcing Expert, and Founder of Victors & Spoils

November: Matt Heusser — Software Testing Author, Professor and Testing Manager

December: James Bach — Software Testing Author, Teacher and Speaker

We have some great guests and ideas lined up for 2010, including software execs, QA thought leaders, and famous journalists & authors.  As always, the goal of Testing The Limits will be to inform, to entertain, and above all else, to help our readers get to know these thought leaders who are worth following and listening to.

Have a suggestion for a future Testing The Limits guest?  Drop us a note or tell us in the comments section.

Continue Reading

Exploratory Software Testing: A Follow-Up Q&A with James Whittaker

James WhittakerLast week, uTest hosted a webinar on exploratory software testing with James Whittaker.  We received a fantastic response from the 250+ attendees, and we couldn’t get to all the questions before our time was up.  Luckily, James was kind enough to sift through a stack of the remaining questions and provide answers to several that jumped out at him.

Also, remember that we’re handpicking five webinar attendees to receive a free copy of his new book on exploratory testing, signed by James.

Q: When making a tour specific to your own application domain, doesn’t that become what is usually called a test scenario? How do you see tours being different from scenarios?

A: Great question and I cover this in my book. Chapter 4 deals with “Tours” and chapter 5 deals with “Scenarios.” In a nutshell, I see scenarios as more prescriptive than tours. Tours are meant as general guidance and scenarios, at least in my mind, are more specific. A tour specifies goals and approach to coming up with test cases, a scenario actually provides an outline of the test cases. Tours leave much more of the actual test case to be constructed as you test. A scenario, in other words, has less variation.

But don’t get caught up in semantics. It’s a continuum of detail really. At one end of the spectrum are fully detailed test cases, at the other is ad hoc testing. Scripts, scenarios, tours, patterns … they all fall somewhere in between.

Q: What is the difference between exploratory analysis and exploratory execution?

A: I don’t like introducing new terms – testing has too many of them already – so I will talk about the concepts here rather than reinforce these exact names. The thought processes that go into exploratory testing are generally considered something that you do while you are executing test cases but this only works with manual testing. In Google’s case, we do substantial test automation and once the test code starts running your chances of introducing exploration is pretty much gone. The automation will execute your test case with brute force and little flexibility.

With automation, you have to do your exploratory thinking up front and this is where we came up with the idea of exploratory analysis. Simply put, the idea is to run the Tours in your head and let your thinking inspire your automation. The best example we have within Google is Rajat Dewan’s example he presented at STAR East and explained on the Google Testing Blog.

Q: Do you have some tips on how to keep testing fresh and new when there is release after release? (To avoid people getting bored and always testing the same things and missing new issues.)

A: In fact, I do. I think this very problem is what I was trying to tackle with the tours. But your question gives me the opportunity to clarify this intent. Static test cases might be fun to come up with and fun the first couple of times you run them but running them build after build and release after release not only gets dull, it also introduces the pesticide paradox. The reality of the situation is that test cases, as a specific physical entity are too low level. They specify a precise sequence of user actions. Tours are a higher level concept and specify purpose and intent and remain flexible on specific input sequences. In this manner a single tour represents any number of test cases.

Now the secret is finding the balance. Some test cases are really important as they once found a bug or they represent an important user-initiated scenario. We want to run these no matter how bored we get. But beyond that, tours allow us more flexibility to increase coverage around the specific test cases and supply the variation that will keep our heads from exploding in boredom.

Q: Can you offer any advice for a developer to be a better partner in the testing process?

A: Indeed. But I want to point out that your questions is asking for advice to devs, not about what test can do to help this partnership (which is the harder answer, so I thank you for that).

I manage a dozen or so projects from cloud to client to back end data center stuff. Some of these have great developer participation and some less so. The devs who are great partners are very involved in testing. They review and provide feedback on our test plans and designs. They become concerned fairly often about whether we are doing a good enough job in test (I mistrust anyone who trusts me and my team too much). They try to steer testers to areas of the product not covered by dev-penned unit tests. They fret more over us finding very few bugs than when we find a lot of bugs (think about that one a moment). They show great interest in what our automation is doing and like to suggest new manual test cases. When they find a bug, they take the time to show it to us instead of just checking in a fix. They invite us to give presentations during all hands and engineering reviews and they take the time to share credit with us when the team succeeds.

I like this question. Maybe I’ll keep thinking about it and make my answer into a paper.

Q: Have you found that your tours work well or help in cases where requirements are sporadic, vaguely defined or non-existent.

A: Having never worked on any other type of project, I can say with some confidence that, yes, they work quite well.

Sincere thanks to James for a great presentation, and to all the attendees for some excellent questions.  We always enjoy seeing discussions about testing be elevated to a strategic level, and so much passion and interest in the subject.  Rest assured, we’ll be scheduling more webinars in the coming year.  In the meantime, you can find a library of free resources about software testing, including eBooks, whitepapers and recorded webinars.  Have other questions for James?  Have suggested topics for future uTest webinars?  Drop us a comment and let your voice be heard!

Continue Reading

Exploratory Software Testing Webinar with James Whittaker — December 10th


Attention uTest Community and prospective uTesters:   If you haven’t registered for tomorrow’s free webinar (December 10th from 1pm to 2pm ET) on Exploratory Software Testing, please click here to reserve a spot.  It’s a hot ticket, with more than 300 testers from around the world already registered to attend.

Many of you have expressed interest in additional resources to help sharpen your testing skills, so this is a great opportunity to attend a free webinar with James Whittaker. He will discuss topics from his new book on Exploratory Software Testing. Additionally, we will be handing out five free copies to attendees (signed by James) – winners will be announced at the end of the webinar.

Hope to see you there!

Continue Reading

James Whittaker on Exploratory Software Testing: A uTest Webinar

For the third time in our brief history, Dr. James Whittaker will be hosting an exclusive uTest webinar. On Thursday, james_whittakerDecember 10th (from 1pm to 2pm EST) James will discuss exploratory software testing – which also happens to be the title of his latest book. Here’s what he plans on covering this time around:

  • How to make test planning more streamlined and prescriptive
  • How to be more conscious about testing and test case design
  • Techniques for helping testers come up with better test cases
  • How to communicate the purpose and intent of test cases

Be sure to reserve your spot now. These webinars are highly recommend for anyone looking to advance their career development, testing skills, or to simply learn more about exploratory software testing. Did we mention that it’s free?

As part of the webinar, there will also be a live Q&A session, and so if you have any questions for James, you’ll be able to submit them online. To get a better idea of what to expect, you should check out his first two uTest webinars: 5 Ways to Revolutionize Your QA and The Future of Software Testing.

Hope to see you there!

Continue Reading