Software Testing Blog

In part II of our interview with Jon Bach, we get his thoughts on testing metrics; common tester stereotypes; the merit of certifications; the testing blogosphere; inventing Twitter in 1986, as well some rapid fire Q&A. We think you’ll like it. By the way, did you miss part I of the interview?

uTest: You have some great tips on how to handle bloated testing numbers and statistics: “Any number, any statistic is like software. It can be tested.” What other tips can you give testers when it comes to having the courage, diplomacy and patience to slow things down and get to the truth?

JB:  For me, the magic words that often make me feel more courageous, diplomatic and patient are: “I have been fooled before.”

No one will argue with that because it’s true.  Scammers often confess that the hardest person to fool is somebody who says “I can be fooled.”  So many times I’ve been so sure I was right just to meet someone who convinced me differently, sometimes in a matter of seconds.  So now say “I could be wrong”, and use other safety language like: “it could be”, “it seems like”, “it looks as if” and “maybe…”  That way, I don’t feel stupid when I’m shown refuting evidence to my claim.  If you practice that, chances are good that you will appear to be a kung fu master who, after having floored your 50th assailant with your skills, slowly backs out of the room on guard for the 51st.

Remember that testing is a craft.  It involves thinking about how things might be different.  Remembering to say “I have been fooled before” is consistent with that spirit.

uTest:  Testing certs: worthwhile or window dressing?

JB:  The only thing worthwhile about them is the debate they provoke.  Window dressing is an apt metaphor because it’s only meant to enhance a window’s *appearance*.  When there’s a flood or a storm or some other strong test of the window, the dressing often gets destroyed. Outside of the flood, people may prefer the look of the dressing; I just want to be a stronger window.  Passing multiple choice tests about so-called “best practices” don’t do that for me.

Read more…

When we published our Michael Bolton interviews earlier this week, we forgot to mention that he’ll be speaking at a bunch of  testing events/seminars in the months ahead. So if you’re in these areas and want to see Michael speak in person (as opposed to Youtube) you owe it yourself to attend.

That said, here are a few upcoming testing events:

• “Why is Testing Taking So Long?” – Michael will be giving an interactive talk on this topic at the Markham meeting for TASSQ, the Toronto Association of System and Software Quality, on Wednesday, February 10, 2010.
• The Conference on Free Testing Tools: Michael will be giving a three-day public offering of his Rapid Software Testing course, and will deliver the keynote talk. The event will be sponsored by the Norwegian Computer Society, and will be held in Trondheim, Norway, March 22-26, 2010.

Read more…

Earlier this week, we published our three-part interview with Michael Bolton.  This was the latest installment in our monthly Testing The Limits series, in which we sit down with luminaries from the worlds of testing, development, crowdsourcing or startup life.  As part of this discussion, we asked Michael for his take on the issue of testing certifications (as we’ve done with Matt Heusser and James Bach in previous months).

In response to what she felt was “cert-bashing,” Charity Stoner of ProtoTest has written a post defending test certifications.  Since we always encourage civil discourse and open-minded debate — and since the purpose of  the Testing The Limits series is to offer up different perspectives from around the world of software — I wanted to shine a light on this post.

What do you think about test certifications?  Do they provide testers with a toolkit that complements their experience and adds real value?  Are they a marketing mechanism that limits what it means to be a professional software tester?  Or is it somewhere in the middle?  I’d love to hear your thoughts.

In the third and final part of the Michael Bolton trilogy, we cover advice for new testers, his hypothetical banishment from Software Land, the blogs he reads and more. Did you miss our earlier interviews? Here’s Part I and Part II.

uTest: Hypothetical: You’ve been banished from testing – nay, ALL software-related activities – for the rest of your days. What will you to earn a living?  What hobbies would you pick up to fill the intellectual void?

MB: Who knows?  For fun, I’d keep playing mandolin, probably. Teach, maybe. Write. I’ve worked in theatre stage management, been a book-keeper, tended bar, worked in a comedy club. In high school I worked in mail rooms during the summer. Whatever I’ve picked up in life, it was because something needed to be done and I was there to do it.  If it didn’t seem like much at first, I started to learn about it quickly. When you invest a little bit of effort to figure out your job, you learn how to makes it faster and better and more interesting. It turns into this great feedback loop. Any job can be more fun when you set out to master it.

uTest: Tell our testing community something about you that your most avid readers don’t know.

MB: While walking through the woods on an island near Vancouver recently, I found myself being quiet and brief, which I like from time to time. Practically nobody knows that.

Lots of people probably don’t know how much I’m eager to help people out. All of my work—courses, articles, conference presentations, this interview—comes with lifetime free technical support. Have a question? Just ask. I might not answer right away—supporting the family with paying work takes precedence over supporting the community—but I’ve never knowingly turned anybody down, so if I don’t answer right away, be persistent. James Bach makes the same offer, by the way. We’ve found that it’s a great way not only to help people, but also to explore problems and come up with solutions and learn things that can help our clients.

uTest: If you were talking to a newbie tester, what advice would you give him or her to set their professional journey off on the right foot?  How about for a 10-year veteran tester?

Read more…

In the first part of our interview with Michael Bolton, we grilled him on the emergence of the Weekend Testers, sensible metrics, Michael Bolton the pop star and a bunch of other topics. In part “deux” of our interview, we tackle the necessity of tester passion, how emotions affect testing, and the greatest threats to the profession. Check back tomorrow for the final segment.

uTest: There’s a lot of passion amongst testing thought leaders about the best way to test, or the best way to manage or train testers.  Often that passion overflows into heated debates.  How can this passion best be channeled to improve the state of testing?

MB: First of all, we should welcome debate, and we should welcome skilled argumentation as part of the art of construction and practice of persuasion. I’ve found, though, that it helps to remember that we’re exploring and challenging ideas. That means it’s good not to get too personally invested in certain ideas, because we’re always learning more, and because changes in context can mean big changes in what needs to be done.

That said, there are some ideas that seem robust for me. I believe that it’s unethical to dumb down people or the work that they do. I believe that we should focus our craft on learning, and learning how to learn rapidly. How can we improve the state of testing? By recognizing that software development is a web of people who are related in service to each other. That means putting people and social issues first. Get that right, and everything else will follow.

Suggestions are cool.  Standards are something else.  No group should be dictating to other people how they must test unless there are compelling human health and safety reasons for it. Do you really believe that the standards people know anything useful about your business? That the force of government-supported regulation, created by busybodies, should weigh on how you do your daily work? And if your answer is No, what are you going to do to get it stopped?

Read more…

We’re thrilled to have Michael Bolton as the latest victim of our Testing the Limits series. As the founder of DevelopSense, Michael has traveled the world teaching the craft of software testing to businesses and individuals alike. Since 2005, he has specialized in courses on Rapid Software Testing – which he co-authored with James Bach. Michael is also a prolific writer, and his publications include hundreds of articles, essays and columns. Aside from his blog, you can keep tabs on his latest work through Twitter.

In Part I of the “trilogy” we discuss the Weekend Testers, testing abroad, how numbers can enslave managers, and of course, his pop-star namesake.

uTest: You’ve been a thought leader in the testing space for a while now, but people still seem to get you confused with Michael Bolton (the singer) on Twitter. Ever thought about creating a tester alias? Or have you considered asking him to change his name since “he’s the one that sucks.” Assuming you (and our readers) have seen Office Space, I bet this joke never gets old.

MB: Yeah, it never gets old.  Try renting a car with this name.

A couple of things on that. First, Office Space captures very well what it’s like to have my name. Second, it’s not his real name; he changed it already. Way back when, before Office Space, I was working in tech support at Quarterdeck Canada.  American callers would occasionally turn north when there were long phone queues in Santa Monica. On one call, when I introduced myself to the customer, he laughed. “Really? That’s your real name?” “Yes, really,” I said, expecting one of the usual jokes. He said, “You know, it isn’t his real name. I used to be his bass player.”  The singer’s real name is Bolotin, but according to the bass player, there was no hope that radio DJs would ever pronounce “Bolotin” right, so he changed it.

uTest: We recently interviewed your friend and colleague James Bach, who had high praise for a group called the Weekend Testers. Can you give our readers a quick recap of what this group does, and whether or not you’re on board with their testing philosophy?

Read more…

In the December episode of our Testing the Limits series, we rapid fire some questions back and forth with James Bach (@jamesmarcusbach).  James is one of the most thought-provoking, outspoken, earnest thought leaders in the testing space.  Check out his blog if you don’t believe us.

Today we’ll be discussing James’ disdain for tester certifications, faking test projects, werewolf hunting in parallel universes and what he would do if he were king (or an angel) for a year. Don’t worry, it’ll all make sense soon. Update: Here’s Part 2 of the interview.

uTest: You’ve been an outspoken critic of traditional certs and classroom education. If you were king for a year, how would you fix testing certifications?  And how would you change a college’s curriculum?

JB: Kings are not powerful enough. I want to be an angel for a year.

You see, certification is promoted by frightened people who feel they need elaborate content-free ceremonies in order to feel competent. But in their hearts they know they are faking it. The fear of being exposed as imposters keeps them from doing much about it. So, in that year I would travel at relativistic speed around the industry. I would visit, by night, the hearts of testers everywhere, giving them inspiration to become excellent at their craft. The ones already certified would wake up and take a long cleansing shower, then write blog posts– by the thousands!– repudiating ISEB, ISTQB, CSQE, and all such blight. They would declare themselves reborn as students of the craft. (The ones not certified will just feel strangely cheerful, at least for testers.)

A spirit of exploration, experimentation, and debate would spread around the industry. It will seem to come from everywhere at once.

Weekend Testers would become Weekday Testers. TMap textbooks would be beaten into plowshares… and then recycled. Test plan templates and TPS reports would blow forgotten through streets lined with cheering crowds playing tester games designed to hone practical reasoning skill. By the thousands! FOR THE WIN!!

As far as university goes, I’ve already been doing my part. I helped found and run the Workshops on Training Software Testers, which brings university professors together to examine how to teach testing better.

I served on an advisory board for the Rochester Institute of Technology when they were trying to set up their degree program in software engineering, too.

But if I were king (not the modern Swedish kind but the old-school Caesar kind) I would make school a lot harder (much easier to expel a bad student) and instead of paying tuition, students would be paid.

Also, there would be no classes, as such, just constant projects and training. In other words, it would be almost exactly like Silicon Valley in the eighties, except with better corporate libraries.

uTest: If a parallel universe where you weren’t involved in testing or software at all – what would you be?

JB: If the parallel universe is before the industrial revolution, then any TWO of the following:

• A freelance sentry.
• A small-time warlord.
• An itinerant geometer.
• Werewolf.
• Werewolf hunter.
• A member of the 1735 French Geodetic expedition, but not the one who got killed by the mob at the bullfight (he had it coming).
• Zorro.
• A gentleman naturalist.
• A buccaneer.
• Gandalf.

uTest: A full day at an ISTQB seminar, or a full day in a college-level computing class – you’re forced to choose one. What’s it gonna be?

Read more…

Today, we finish up our “Testing The Limits” interview with Matt Heusser.  Be sure to check out part 1 of this interview.  In this installation, we’ll discuss which mobile app testing, his OS of choice, and the testing blogs and sites Matt reads.

What other blogs, sites or message boards do you read to stay on the leading edge of all things testing?
MH: I’ve got a bit of a bias there, as I write a monthly column for Software Test and Performance Magazine. You can get the PDF version for free every month. I also subscribe to Better Software Magazine. For communities, I like softwaretestingclub.com and the agile-testing discussion list. For blogs, well, there’s James Bach, Cem Kaner, and Michael Bolton. Adam Goucher has been doing a lot of writing lately, including editing the new book, Beautiful Testing, which I contributed a chapter to. And I spent a fair amount of time working on my own blog, “Testing At The Edge of Chaos.”

Recently I’ve been getting to know people by working on projects with them; my friend Chris McMahon started a Google Group on Writing About Testing which I found to be a blast. Through that group (and the Miagi-Do School) I’ve met quite a few new bloggers: Markus Gaertner, Lanette Creamer, Marlena Compton, and Ajay Balamurugadas come immediately to mind.

What OS are you running right now? What’s your browser of choice? Anti-virus? Inquiring minds would like to know.
MH: Max OS X, and I have to support all of them, so I run all browsers. Anti-Virus? Dude, I told you, I use a mac, and I try to avoid the questionable websites that host the viruses. What have you been browsing lately, Mr. Johnston?  (Ed. Note: I’ll ask the questions around here.)

Read more…

Earlier this week, several members of the uTest team took in the opening reception of Software Test & Performance’s STPCon event.  We overheard some great conversations, and even jumped into the fray a few times ourselves.  It was a lively crowd and a great venue (the Hyatt Regency) right along the Charles River.  The event continues today, but even if you’re not attending you can still follow along, check out the STPCon Twitter stream here.

This event gave us a chance to reconnect with our friends from STP Collaborative, as well as sit in on sessions from top-shelf testing thinkers like James Bach, Jon Bach, Michael Bolton and Matt Heusser.  We also got to connect with about a half dozen local uTesters who took us up on our invitation to attend the reception.

We had a few people from uTest in attendance on Thursday and they gave the presenters high marks overall.  The two that I heard the most comments on, however, were from James Bach and Michael Bolton.  Bach tackled the provocative subject of “How to Fake a Test Project.”  Bolton took on the topic of “Rapid Software Testing.”  We’ll see if we can get either of their presentations and share them here. Also of great interest was Friday morning’s SpeedGeeking session with Matt Heusser, which put the speakers on the spot, giving them only five minutes to get right to their point. Talk about fighting your natural instincts!

For anyone who attended, chime in and share your most or least favorite moments from STPCon?  What surprised you, frightened you, entertained you or just generally pissed you off? Sound off in the comments or drop us a line.

Meanwhile, the uTest paparazzi was present and ready to capture some of the scenes from the evening.

Matt Heusser and James Bach take a timeout from their testing debate for a quick photo

Matt Johnston and James Bach deep in discussion about the art & science of testing (and the difference between the two)

Jennifer Moebius and STP chief, Andy Muns take a break from the exhibits to pose for a pic

Calling all New England QA and software testing professionals!

We will be co-hosting a free tester meetup with STP (Software Test & Performance) as part of the kickoff reception for their big event, STPCon 2009 at the Hyatt Regency Cambridge.  This meetup will be Wednesday, October 21 at 5:30pm.

Join us for a great evening of networking that will be held in the STPCon exhibits area. There, you’ll have the opportunity to connect with your peers, connect with execs from uTest and STP, discover new products and features and talk to the experts who created them.

Another great perk for attendees is that you’ll have the opportunity to discuss the latest and greatest trends with industry leaders such as James Bach and Michael Bolton.

If you’re around, it would be great to meet you in person!  To register, please visit: http://utest2009stpcon.eventbrite.com/.