Software Testing Blog

In the third and final part of the Michael Bolton trilogy, we cover advice for new testers, his hypothetical banishment from Software Land, the blogs he reads and more. Did you miss our earlier interviews? Here’s Part I and Part II.

uTest: Hypothetical: You’ve been banished from testing – nay, ALL software-related activities – for the rest of your days. What will you to earn a living?  What hobbies would you pick up to fill the intellectual void?

MB: Who knows?  For fun, I’d keep playing mandolin, probably. Teach, maybe. Write. I’ve worked in theatre stage management, been a book-keeper, tended bar, worked in a comedy club. In high school I worked in mail rooms during the summer. Whatever I’ve picked up in life, it was because something needed to be done and I was there to do it.  If it didn’t seem like much at first, I started to learn about it quickly. When you invest a little bit of effort to figure out your job, you learn how to makes it faster and better and more interesting. It turns into this great feedback loop. Any job can be more fun when you set out to master it.

uTest: Tell our testing community something about you that your most avid readers don’t know.

MB: While walking through the woods on an island near Vancouver recently, I found myself being quiet and brief, which I like from time to time. Practically nobody knows that.

Lots of people probably don’t know how much I’m eager to help people out. All of my work—courses, articles, conference presentations, this interview—comes with lifetime free technical support. Have a question? Just ask. I might not answer right away—supporting the family with paying work takes precedence over supporting the community—but I’ve never knowingly turned anybody down, so if I don’t answer right away, be persistent. James Bach makes the same offer, by the way. We’ve found that it’s a great way not only to help people, but also to explore problems and come up with solutions and learn things that can help our clients.

uTest: If you were talking to a newbie tester, what advice would you give him or her to set their professional journey off on the right foot?  How about for a 10-year veteran tester?

MB: To both: learn, and don’t stop learning. Practice in your work, but also on open source projects, and in your life generally. Study testing, but note that the great ideas are likely to come from outside the field—at least the narrow vision of the field that the process enthusiasts and “certificationists” present. Testing is a marvelously interdisciplinary craft.  One implication is that whatever you bring to the table from your life and your experience and your education can inform new ideas about testing.  Join or start communities of skilled testers in your area. Join online groups; I like the Software Testing Mailing List and the Software Testing Club particularly. Learn about your craft and build your reputation by asking questions and seeking answers. As with testing itself, the really excellent work starts with asking questions.

uTest: In your opinion, what characteristics, skills or experience separate a good tester from a great tester?  How about in a testing manager?

MB: My colleagues and I have been talking about skills of great testers lately.  (In term of public postings, James and Jon Bach have podcast a nice pair of conversations which you can find here. James also did a CodingQA podcast here. One key skill is the ability to identify context quickly, which helps to avoid all kinds of pathologies—wasting time, failing understand the mission, testing the wrong stuff, investigating unimportant risks, and so forth.  Get the context right, and you’ve got a better shot at doing excellent testing. We teach people to consider and ask questions about the test lab, the test team, the development model and the programmers, the requirements, and the mission to help with that.

Another skill—which we also teach—is something that James has named test framing, which is the same kind of thing in the other direction. It’s the high-level skill—a skill set, really—associated with linking the context, your motivation, the actions you perform, the observations you make, the inferences you draw, and the story you tell about all that stuff.  It’s the ability to follow and describe the threads of logic forward and backward, from the mission to the outcome of the test and back again. It’s the ability to competently address and answer the questions “How, specifically, does this test relate to your mission?” as you prepare to test and “How does the outcome fulfill your mission?” after you’re done.

We’ve seen some good testers whom we can’t quite call great, because they have a hard time with some aspect of that skill set. They seem to lose the ability to link premises and conclusions at some point.

I think there’s a strong relationship to the skills of a great test manager there. A great test manager needs to be able to do the same kind of framing with respect to the test project and the test strategy. A manager that can frame the project can negotiate for people and resources more effectively, can train and deploy her people more efficiently, can explain the work of her team, can defend herself skillfully, can adapt to rapidly changing circumstances. That’s really important, because most projects are messy and confusing. If they weren’t we wouldn’t need testers.

Here’s an example: Test managers get asked questions like “When is testing going to be done?” That question is hard for some test managers because it’s not the test manager’s job to make the decision. They feel pressured to provide an answer, but they can’t frame the problem. A great test manager can deal with that pressure. First: Problems in a product get introduced almost entirely by people other than the testers. Second: Neither those people nor the testers know what problems are there, or where they are. Third: Testing is done in cycles that reveal as-yet unknown information about problems that may or may not need to be fixed. Fourth: The decision to fix the problems depends on the programmers and on the product owner. Fifth: Testing isn’t done until the problems in the product are fixed to the product owner’s satisfaction.

If those things are true (and they are), the amount of investigation and reporting and bug verification testers have to do is unpredictable until you’ve actually tested the product, made decisions about what’s to be fixed, and fixed the problems. Well… it’s predictable, but the prediction doesn’t have any real validity, because there are too many variables that could throw the whole prediction off. “Accurate” predictions in this business come mostly self-fulfilling prophecy—and from ignoring changes in scope or staffing or schedules or budgets or tools that just happened to make the prediction come true. If you set only a couple of data points for your targets, all the other stuff can change without people noticing much.

So testing cycles can be scheduled, but testing cannot be done until the last problem deemed important by the product owner is fixed, by the programmers, to his satisfaction. The great test manager can help the product owner to identify and decide upon what information he needs to know about the product. His goal is to be satisfied in his technical awareness of the product—in relation to the business needs—such that it makes business sense to ship or deploy the product. The test manager and the product owner collaborate in identifying the testing tasks that might help in providing that information.  That turns into the mission for that cycle of testing, test manager and her team can then set about fulfilling the mission—and adapting it when the product owner changes his mind, or when other things don’t go according to plan.  But in the end, the question of when testing will be done depends very little upon the test manager, and far more upon the programmers, the product owner, and a bunch of information that is presently unknown.  A great test manager can explain that in a way that satisfies the product owner, without being trapped into an unachievable commitment.

uTest: At STPCon, you also said that “Testing can be assisted by automation, but automation is not the arbiter of value. We are.” Does this mean that testing will never be replaced by machines? And does this include cyborgs?

MB: Automation is a tool; it’s a means to an end.  It’s a medium, in the McLuhan sense—it extends or enhances or enables or accelerates or intensifies some human capability, but it doesn’t replace that capability. Writing and email and wikis extend our ability to communicate, but they don’t replace talking.

Machines can’t make decisions about what’s best for us, because they’re only extensions of some human faculty. Value is about what humans want, what they like, what they’re willing to pay or do. People don’t even like other people making those decisions for them.

uTest: You’re an active blogger, teacher and speaker – do you enjoy these things more than testing itself?

MB: That’s a little like asking about someone’s favorite Beatle album. I love testing, I love teaching it, I love talking about it. It’s a feedback loop. I wouldn’t be able to do the other stuff if I didn’t test, and the more testing and research about testing that I do, the more I find that’s interesting to talk about. When we teach Rapid Testing, we offer a free day of hands-on testing and coaching. We offer that separately too, and it’s always a source of great learning and great fun.  I’d be delighted for clients to involve me more directly more often.

uTest: Who do you read in the world of testing (blogs, sites, journalists, et al)?

MB: I have a very expansive view of the world of testing. To me, the world of testing is the world of human knowledge and inquiry, of critical thinking, of science. That includes software-related stuff, but it includes a ton of other disciplines too:  medicine (Jerome Groopman, Atul Gawande), aviation (Patrick Smith on Salon.com), education and other social issues (David Cayley), advertising and marketing (Terry O’Reilly), economics (James Surowiecki, John Cassidy, Herbert Simon), security (Bruce Schneier), risk (Nassim Taleb).  All those folks and their specialties are Googleable. I really like Malcolm Gladwell’s writing, too; he’s harder to pigeonhole, but he’s good at finding intriguing sides to the basic story. I first encountered most of the writers via The New Yorker and most of the broadcasters via CBC Radio. I like starting with very accessible stuff, and then if something interests me, I follow the lines of references, bibliographies and notes.

A couple of testing books that came out recently have few notes and no bibliography. That’s like publishing malpractice, so far as I’m concerned. You’ll notice that I’ve mentioned a lot of names already. That’s specifically so that people who are reading this interview can check out the stuff that’s fascinated me and helped to spark new ideas. I hope it helps to inspire people to talk about their own discoveries. Thank you for this opportunity to chat about mine!

Editor’s note: We hope you’ve enjoyed our three-part chat with Michael Bolton. If there’s a question you wanted us to ask that we didn’t get to, you can ask Michael directly at: michael@developsense.com.

Have suggestions for our next interview? Send those along as well!