Software Testing Blog

Today, we finish up our “Testing The Limits” interview with Matt Heusser.  Be sure to check out part 1 of this interview.  In this installation, we’ll discuss which mobile app testing, his OS of choice, and the testing blogs and sites Matt reads.

What other blogs, sites or message boards do you read to stay on the leading edge of all things testing?
MH: I’ve got a bit of a bias there, as I write a monthly column for Software Test and Performance Magazine. You can get the PDF version for free every month. I also subscribe to Better Software Magazine. For communities, I like softwaretestingclub.com and the agile-testing discussion list. For blogs, well, there’s James Bach, Cem Kaner, and Michael Bolton. Adam Goucher has been doing a lot of writing lately, including editing the new book, Beautiful Testing, which I contributed a chapter to. And I spent a fair amount of time working on my own blog, “Testing At The Edge of Chaos.”

Recently I’ve been getting to know people by working on projects with them; my friend Chris McMahon started a Google Group on Writing About Testing which I found to be a blast. Through that group (and the Miagi-Do School) I’ve met quite a few new bloggers: Markus Gaertner, Lanette Creamer, Marlena Compton, and Ajay Balamurugadas come immediately to mind.

What OS are you running right now? What’s your browser of choice? Anti-virus? Inquiring minds would like to know.
MH: Max OS X, and I have to support all of them, so I run all browsers. Anti-Virus? Dude, I told you, I use a mac, and I try to avoid the questionable websites that host the viruses. What have you been browsing lately, Mr. Johnston?  (Ed. Note: I’ll ask the questions around here.)

As more companies develop mobile apps, mobile testing is becoming a greater concern. Are you seeing a convergence of mobile app testing with traditional software testing? Or is mobile testing an separate entity altogether?MH: My initial thought is that testing mobile apps is a nice little niche inside of web-based software testing. But mobile devices keep getting more and more standards-compliant, and thanks to sites like www.iphonetester.com, you can test handhelds without physically owning a dozen different makes and models. So while I believe there is value in identifying yourself as a mobile device tester, I suspect that value is going to decrease over time.

What attributes and skills make for the best software testers?
MH: My experience lately is with web-based and database-backed commercial and transactional applications, and so I’m going to give an answer that assumes that world view. First off, I think good testers understand the business itself – what the basic transactions are, and what happens if they go wrong. The strong tester will use this knowledge and seek out complexity and undefined behavior; bugs love to hang out there. Strong testers are good systems thinkers; they can perform modeling of the system to figure out where the weak points are. They have a strong memory; they can track a large number of variables in their head at one time. They may play chess, soduku, or maybe just board games by Avalon Hill, or possibly Steve Jackson Games in it’s prime. James Bach pointed me to curiosity, and I think he’s right – if you never ask “what happens when I click this?” it will be hard for you to click this. They’ll want to know common failure modes across all software, and will want to study “quick attacks”, or the “How to break … Software” literature. They will be solid critical and systems thinkers – the kind of skills you can get from listening to politicians and asking what they are not saying, or reading articles in the press and looking for the bias. They will have solid middle school math skills, but be able to figure out a 15% tip quickly, combined with a little bit of statistics and probability. An ability to do some programming, to understand HTML and javascript and such really helps.

I would suggest influence skills combined with a healthy dose of skepticism. It is very hard to find someone with both of these, but to find the problem you need to be skeptical of glib answers and false confidence – and to get anything done about it, you’ll need influence skills. Influence is more than conversation, it is written communication and situational awareness.

Another hard to measure skill is creativity, but it is so crucial. The industry may like to believe that test cases can be generated automatically by some standardized process, but the reality is someone has to think of them, and a creative mind can find all kinds of possible failures that a dull one will miss.

Finally, there’s discipline, which is an interesting one. I have to admit, I have a hard time denying myself as much as a cup of coffee. I’m not very good at forcing myself to testing – I actually want to test. If you’ll permit me just one brag, I think it is the actually wanting to test part that makes me good at it, not the discipline.

Of course it’s going to be different by the job, government and regulated work is very different than mass-market commercial software. And some organizations care a lot about technical skills like dotNet or Java or whatever. You might have a very effective boss who covers for a tester who lacks influence skills. If I had to pick one tester attribute to spark the rest, it would be curiosity. If you just don’t care “what would happen if?” you wouldn’t imagine it, and you won’t press the button to find out.

You seem to be big on tester training lately. From what you’ve read and observed, what’s the best way for a new tester to get up and running?
MH: I’m afraid I can’t give a simple answer like “just go to a conference for a week” – it would have to customized by the environment. So you’d start with some sort of exercise to look at where a tester is according to a skills inventory, where they fit in with the team, and what to work on next.

Likely, that would be a combination of job shadowing, directed readings, brown bag lunches, mentoring, and, possibly, external training. I do think sending a new tester to a conference is a good idea, mostly because of the rich variety of training options and idea exposure they will have that week. A conference offers an opportunity to customize training just by selecting which sessions to attend. If I didn’t have the budget for training, and wanted something local and online, I would consider something like the black box software testing course run by the Association for Software Testing.

Whatever the new tester does, I suggest it be some combination of theory and practical exercise, with the students aware of the exercise and built-in time for reflection on what happened.

Now, what do I recommend if you have a certain budget and want to train an entire team? Something customized in a similar way. So, for example, I am getting more and more excited about the idea of bringing the trainer on-site for a day or two of consulting by walking around. Based on what he observes, the trainer develops a day or two of custom training using similar technologies to those used in the actual workspace; then he comes back for a two days of actual testing, followed by a retrospective. The trainer might then come back every month or two for several brief engagements.

The idea there is to actually tie the training to the work being done, to move it from “Drive-by training” to “helping to collaboratively explore how to improve our work.” Or, in other words, to do outcome-based training. I know. Crazy.

Are great testers born or bred – asked differently is being a world-class tester something that anyone can learn or are you born with it in your blood?
Can I answer that by telling a story? Malcolm Gladwell has a new book out, called Outliers: The Story of Success. In the book he discusses the nature of language – that some cultures, such as South Korea, are high-ceremony cultures. These are the places where the conversations between superiors and subordinates is very formal and scripted. He also discusses low ceremony cultures, like the United States, that have more of a cowboy culture. Gladwell points out that, while human beings make mistakes, the high ceremony culture offers no way for a subordinate to point the mistake out and attempt to correct it. While an American co-pilot might say “we’re flying into a storm you ninny”, in other cultures that behavior is not allowed. That’s no big deal when you’re talking about how many lumps of sugar you’ve counted out, but when it comes to flying airplanes – or mission and life critical software – it matters. So I would say that culture and values or ideology, plays a huge part in the effectiveness of the testing. So much of our work is shining the light of truth when it would be more politically palatable to not notice.

So on the one hand we’ve got cultural values, and on another critical thinking, modeling skills, and curiosity. Certainly critical thinking and modeling can be strengthened through exercise. Curiosity? That’s a good question. You might make an argument that curiosity is inherent, that it “comes with the genes” so to speak — but I’ve seen enough children have their curiosity sucked out by schooling to believe that it must be possible, somehow, to create an environment to encourage it to come back in.