Software Testing Blog

In the second part of our Testing the Limits interview with James Whittaker, we tackle Google vs. Microsoft; dogs vs. cats; why SCRUM is just a name; his advice for college graduates; bad habits of exploratory testing and more. If you missed Part I, you can find it here.

If you want to read more of James’ work, bookmarking the Google Testing Blog would be a good place to start. You can also read his 2009 book Exploratory Software Testing or check out some of his uTest eBooks and webinars.

uTest: The Microsoft vs. Google battle continues to play out very publicly in the media. Just last week, Computerworld wrote this story: “Microsoft: No Matter What Google Says, Windows Is Secure.” Having been at both companies, we think you have a unique perspective on this one. Any thoughts?

JW: Let me say right away that I enjoyed my time at Microsoft and admire the company and the smart people who work there. As a resident of Seattle, it is in my best interest for Microsoft to prosper! But the two companies are vastly different regarding the way their talent is managed and their products are built. Google is an engineering-centric company where innovation comes from individuals who are empowered to do whatever they need to get ideas into production. Much has been made of Google’s game-theory approach to managing people where rewards are given quickly for impactful behavior. It works. Morale is high and people work very hard and take quality very seriously.

Does this mean we produce more secure or more reliable products? We try hard to do so; Microsoft tries equally hard. I think we have the advantage of less legacy and a more modern and reliable platform (the Web as opposed to client operating systems) to work from. But the secret sauce at both companies is the same: hard work and due diligence.

uTest: You shared with us (as the pioneer of Testing the Limits posts) that your first assignment at Google was “To raise the level of testing precision and diligence.” So, how did it go?

JW: It didn’t take long. Google was mostly already there so I can’t really take credit for it. Now I am busy raising the bar further.

uTest: Top tester Glory Leung is curious: What are your views on SCRUM testing in general? Are people doing it properly? What is the ideal situation?

JW: Scrum is just a name. I don’t like names, they feel too confining and people have their own ideas of what they mean. I took a lot of flak for using the name ‘exploratory testing’ for my book. People love to confine you to how they view a specific named idea or technique. Flexibility is required.

The ideal situation is that the best testers for the job be involved. For unit testing and TDD this means the individual devs who wrote the code. No one is more qualified to test a function in isolation than the person who wrote it. However the dev is often the worse person to test how his function interacts within a larger feature or with its operating environment. At Google we expect devs to do the former, all of it. Then and only then have they earned the right to have testers do the latter. Having broken or missing unit tests is a great way to ensure that testers get taken off your product.

The question I am struggling with now is product expertise. What is the right mixture of testers who are close enough to the product to become real experts with it and specialist testers who are expert breakers? Call it whatever you like but product expertise and testing expertise are the two ingredients that I find make for good testing, regardless of your ship cycles or how your team develops their code.

uTest: There are many varieties of “testing frameworks” out there that guide testers in choosing test cases and defines the way they approach test design. Out of the ones you’ve recently seen (e.g. Input Domain, Divide & Conquer, Fishbowl, Storybook, Pessimists), which of these works best? Is it a combination of two or three? Does it depend on the job?

JW: More names, less meaning and more chances of some consultant claiming ownership over them! Like I said in the previous answer to me the thing that I find most valuable are people who know the product and people who know how to test. Stir those together and you have a winning formula.

uTest: Doc JW rapid-fire (in no particular order):

1. Last book read? The Hobbit with my 12 year old son. It was his first time (how I envied him for that!)
2. Last movie watched? Avatar, 4 times.
3. Favorite band or album? Neil Young, Led Zeppelin, and Nirvana … in that order.
4. Favorite Kevin Bacon movie? Kevin who?
5. Browser of choice? Chrome!!!
6. Cats or dogs? Dogs, not even close.
7. Favorite video game? I played Monkey Ball 14 years ago, once.
8. Which mobile device are you packin’? Android Nexus One
9. Favorite mobile app? Google Sky Map.

uTest: What would you be doing with your career if you were not a Test Director (e.g. bullfighter, gourmet chef, cobbler, explorer)?

JW: Stand up comedian. I still intend to do it some day.

uTest: Can you give us a sneak peek into any of your upcoming guest lectures, books, articles or presentations on the horizon?

JW: Alan Page inspired me with his book on how Microsoft does testing, perhaps I will write one on how Google does it. There is almost no overlap. I have also doodled a memoir of sorts trying to figure out how a guy like me managed to succeed. I’ve only shared it with a single reader who thinks I should wait until I retire to publish it.

uTest: It’s graduation time…  what would you tell college students who want to enter the world of professional software testing?  What should they do (or avoid) to get their career started on the right track?

JW: Simple, learn computer science. Then, get a job at a company that takes testing seriously and be prepared to learn. A degree in CS is, for me, the first and most important background.

uTest: Since you literally wrote the book on the subject, what separates a bad exploratory tester from a good exploratory tester?  And what’s the difference between good and great in this area?

JW: Bad exploratory testers spend little time preparing and are only seeking bugs as their primary outcome. Good exploratory testers apply structure to their work and use tools to collect data so that their testing sessions are more meaningful than just bug finding exercises. Exploratory testing should be connected to the rest of the testing process and not just a late cycle bug finding activity.

Editor’s note: We hope you’ve enjoyed our latest interview with James Whittaker. We’re always open to suggestions for future guests, so if you have someone in mind, or want to submit a question of your own, be sure to let us know!