Software Testing Blog

In part II of our interview with Scott Barber, we tackle the so-called “consensus” among the Context-Driven School; his start as a software tester; why software problems are mostly people problems; reporting out-scope-bugs and much more.

We will post the final installment tomorrow. By the way, did you miss Part I?

uTest: There’s a lot of consensus among the Context-Driven School of Software Testing, but are there any fundamental differences? You can’t agree with guys like Michael Bolton and James Bach on everything, can you? Speak freely, your secrets are safe with us!

SB: I don’t know that there *is* a lot of consensus. I know that I am very much Context-Driven. In fact, my license plate says “CONTEXT”. That said, the buzz and counter-buzz surrounding the Context-Driven School of Software Testing is still causing as much harm as good.

I think the underlying problem is that the creation of software is widely treated as a software development activity, which may or may not benefit from related activities like testing. I find this thought process just plain silly. Software development, in my experience, is done for one of four reasons; altruism (minimal), education (some), research and development (slightly more than some), and/or financial/business advantage reasons (most). I think the fact that software is created for different reasons and purposes makes software development at least context-dependent by definition. The difference between being context-dependent and being context-driven is fundamentally the difference between being reactive vs. being proactive.

I think there never would have been a need to call out or name the Context-Driven School of Software Testing if testing hadn’t been so undervalued, misunderstood, and too often ignored entirely for so long that some entrepreneurial folks decided to step up and start talking about “the right way” to test software. While I’ll be one of the first to admit that many of these folks published “right ways” that were far superior to the state of the practice of the time, none of these “right ways” were right for everyone or every situation.

So, in an attempt to help people and organizations be aware that there are many ways to approach testing, and that the way that is “best” depends on a wide variety of factors, none of which involve which entrepreneurial person has the best “sales pitch” for their favorite approach.

Regardless of what the buzz may say, context-driven isn’t about Exploratory Testing, isn’t against test scripts, and doesn’t dictate who do what testing when. The way I talk about being driven by context is to regularly stop and ask yourself, “What can I do, right now that will be of most value to my team/project/company?” If you make decisions that way, instead of simply doing what someone who knows nothing about your situation recommends, I’d say that is at least the first step to being driven by context.

uTest: We’re always interested in how people got involved in software testing. What made you choose this path – what lit the spark?

SB:  The short story is that I didn’t choose software testing, it chose me.

I was dabbling in various technology and software related jobs after my time in the Army, when the company I was working for got bought by a big company who changed our compensation plan to something I couldn’t live with. While complaining to my friend, he said “Forget them, come work for us, we need a Performance Test Engineer.” In response, I asked “What’s that?”  He said “Don’t worry, you’ll like it.”

I started the job a few weeks later, and he was right, I did like it. I liked it so much, in fact, that I’m still happy being a tester over 10 years later. And to think, when I started that job I wasn’t even aware there were people who tested software for a living.

uTest: Follow-up question:  If you weren’t in testing, or anything related to software, what would you be doing right now?

SB: That’s hard to say. I went to college to become a Civil Engineer, and while I did very well in school and love the field, I was already too far removed from it by the time I left the Army to have a viable path back in. I’ve considered teaching secondary school or at a university, but there are aspects to those careers that I don’t think I would end up being very good at or enjoying. I’ve considered becoming the editor for a technical magazine, and becoming a conference coordinator for a technical conference. I’ve even considered going to work on the administrative side of a non-profit whose work inspires me.

My friends who know me well are always telling me that I should “retire into” sports announcing and/or coaching. I don’t know if that’s even vaguely possible, but I do know that there is nothing I seem to enjoy doing more than coaching and impressing people sitting around me by constantly pre-empting sports casters while watching a game.

uTest: Say we choose a software testing project at random. You don’t anything about it, except for the fact that there’s a major bottleneck somewhere in the process that is preventing quality software from getting released on-schedule. If you had to guess, what would the source of that bottleneck most likely be in a typical 2010 company? Management? Testers? Technology? Metrics? Give us your best educated guess and why.

SB: I believe it was in his book Secrets of Consulting that Jerry Weinburg stated “Every problem is a people problem.” Throughout my career, I have found this to be a great place to start. People make decisions, people devise and implement process, people write and test code, and people manage projects. If there is something preventing software from being released on time, it’s because someone made a mistake, a poor decision, an inaccurate estimate, or simply isn’t doing their job. It may or may not be relevant to figure out who did what that lead to the problem, but if you trace a problem far enough, eventually you end up at a person. The way I see it, people problems fall under management.

Add to this that quality education for managers and executives overseeing software development is at least as sparse as quality education for testers, and it makes “Management” a pretty high percentage place to start.  Of course, it’s also the most difficult to do anything about, so frequently organizations try to address their software development challenges by attacking symptoms rather than the cause.

uTest: Often times, testers will report bugs they deem “critical” (or high priority, P1, or whichever vernacular you choose) that are subsequently rejected by the client, product manager or dev lead as being ‘out of scope’ or ‘works as designed.’ What advice do you have for testers trying to make the case that a particular bug MUST be addressed before launch? Has this ever happened to you?

SB: It used to happen to me all the time, until one day I realized that I’m a tester, not an executive. My job is to provide information that my team, my employer, and whatever other stakeholders there are for my project want or need to know to make informed decisions. My job is to help them understand the state of the software and the possible implications of what I’ve found.

If, as a tester, you feel that a bug you have found is “extra important” to fix prior to launch, I recommend getting an audience with a decision maker and explaining the bug and its implications to items that matter to them (such as profitability, public relations, etc.) until you are convinced they can accurately represent your side of the argument to their peers and supervisors. After that you have two choices, as I see it, accept the decision that is made as being best for the business based on information that you’ll probably never have, or decide that your values and your company’s values are not in line and start looking for a job in a company with values closer to your own.

uTest: Favorite book that taught you an important testing lesson (even if the book wasn’t about testing per se)?  TV show?  Haiku?  Parable?  Off-Broadway play?  Shoot, just share any source of testing wisdom that has inspired you.

SB: General Systems Thinking by Jerry Weinberg, as I mentioned earlier. To be honest, it validated the way I have always thought about testing more than actually teaching me something new. That said, I’ve been teaching testers and other members of software development teams for almost 10 years and I can say with high degree of confidence that if all of those folks were reasonably skilled in systems thinking, software systems and their development, it would be in a much better place than it is today.

On a more pop-culture note, the television series House, M.D. is absolutely the best example of testing I’ve seen on a T.V. or movie screen. This is a show about diagnostic medicine, not software, but these people know testing. They understand the difference between positive and negative testing. They use scripted and exploratory testing. They aren’t satisfied with minimizing symptoms, they have an inner need to determine and eliminate root causes of unacceptable symptoms or behaviors. In fact, I’ve been influenced enough by the show that in one of my training classes, I have a segment titled “If Dr. House were a Performance Tester”, I’ve considered starting a blog under the same title, and I’ve seriously considered trying to popularize “Software Testing and Diagnostics Department” as a replacement for “Software Quality Assurance Department”.

Editor’s note: Check out Part III of the interview. .