Software Testing Blog

Our Testing the Limits guest this month is testing guru Scott Barber, the Chief Technologist of PerfTestPlus. A speaker, writer, teacher and entrepreneur, Scott has one of the most impressive resumes in the business, particularly in the realm of customized testing methodologies, embedded systems testing, personal security systems and other topics – all of which are discussed on his blog.

In Part I of our 3-part interview, Scott discusses the Manifesto for Agile Development (almost ten years after it was created); the expectations of today’s testing managers; the notion of testers as an “unfortunate necessity”, the 1983 War Games movie and more.

uTest: As a signatory on the Manifesto for Agile Development, can you comment on the progress being made by software companies in upholding these principles? Have they exceeded your expectations, or is there still a long way to go?

SB: Honestly, I think the buzz around the “Agile movement” has, in many cases, taken the industry in an unfortunate direction. I meet far too many people and companies who are completely unfamiliar with the Agile Manifesto and think of Agile as a collection of practices, processes, and tools. The reality is that Agile is a far more a mindset and a culture than it is a collection of practices, processes and tools. Agile isn’t the best fit for every situation, or for every person.

I believe that the trend to “go Agile” is misguided. If a company is developing good software, the people involved in developing that software are happy working there, the software development is sustainable, and the business is being adequately served by that software, there’s really no need for them to try to be more or less Agile. Agile has challenges like any other culture, but the single biggest challenge I find is companies trying to solve development, process, management, and/or schedule problems by “going Agile.” Teams who have grown up in a culture that is fundamentally different than Agile simply will not find it easy to “go Agile.”

Think of it this way. Telling a team to “go Agile” between projects is like telling a professional baseball player to switch from being a right-handed hitter to being a left-handed hitter between seasons – and expecting that to break their hitting slump. You simply don’t see that. What you do see is batters working with coaches to re-evaluate their swing, work on their timing, spend more time in the weight room, etc.

Personally, I am happiest when I’m working in an Agile culture, but that is a personal preference, nothing more. What I do think would serve individuals and teams well, would be to read the Agile Manifesto and really think about whether or not those principles are a good fit for themselves, their team, and/or their organization. If so, I recommend embracing them.  If not, I recommend finding or developing an equivalent set of principles to build a culture around.

uTest: In a recent article, you highlight the critical point that “the perceived value of testing varies widely from individual to individual, team to team and company to company.” Can you further elaborate on how testing managers can do a better job of setting clear expectations?

SB: Consider these scenarios:

• Company A executives are looking for the testing to be their defense in case of failure and lawsuits
• Company B project managers want testing to say “ship it” so that if there are problems in production, they don’t get in trouble
• Company C developers want testing to show them the issues before management notices them and mandates changes to the development process… yet again.
• Company D is developing software for a client who expects testing to ensure their needs are met
• Company E just spent millions on a marketing campaign and is counting on testing to ensure that the company doesn’t get egg on its face when the campaign is released.

In each of these cases, the test manager is certainly the person who needs to lead the test team to adding the most value to the project. The challenge is that the test manager is often in the worst position to get straight answers.  Test managers are typically the lowest-level role in a company to hold the title “manager.” They don’t generally get to sit in the board meetings where defense against lawsuits are discussed, they don’t frequently have direct contact with the client, developers don’t often treat test managers as confidants, and who would tell a test manager that the purpose of his/her team is to take the blame if something goes wrong, even if it is completely out of their control?

How test managers, and testers, can do a better job is to take a step back and really consider what is going on in the company, what instructions they are being given, what questions they are being asked, and who seems happy or grumpy with the test results being provided.  Paying attention things like those will frequently make the “real story” pretty clear, and once the story starts to become clear, you can start asking smart questions that actually lead to pretty straight answers.

What doesn’t work is assuming that because you are the test manager, or the tester, that you *know* how your testing adds value to the company, that you *know* what the most important testing is, and that you *know* the managers and executives are idiots because of what they are telling you to test.  The fact is that even if your managers and executives are idiots, that doesn’t mean that they don’t have a sound and reasonable business reason to want what they are asking for. It’s not up to us to judge them for either being idiots or for making clumsy requests that don’t actually help them resolve their concern. It is up to us, however, to help them figure out what they actually need, help them figure out how to meet that need, and then guide our testing to best serve that need.

uTest: You have an impressive knack for drawing testing analogies from your lessons learned as an officer in the U.S. Army. What is the true value of outlining the situation, mission and intent of a software development/testing project?

SB: Thanks.  It’s not just the Army; it’s my life experience in general.  I suspect that has more to do with the fact that my parents were both educators and that I didn’t even know there was such a thing as software testing until I got hired to be a “Performance Engineer” and was introduced to the software test team on my way to my first client. So I pretty much had to figure it all out for myself with no real training in testing.

In this case, I was trying to figure out what my role really was. I clearly was not the decision-maker about whether or not to ship the product. I didn’t have the authority to tell the developers what should be fixed. So I applied the model I knew and asked myself “What is the situation here?”, “What is my mission?”, and “What is the overall intent of the client for the project as a whole?”

Over time, what I found was that the situation question helped me to understand things like where testing fit in the scheme of the project; who is panicked over what; how long I have to complete my tasks with what resources; who my allies are; what things are likely to block my success – that sort of thing. Much later, I learned that many testers refer to this as “context”, which I think is a very good word for it.

The mission question helped me figure out what was expected of me.  As it turned out, the folks I was reporting to often didn’t really know how to respond to my question of “What is my mission?”  They’d say things like “Well, test it and give me the results, of course.” My follow-up question of “Test it for what?” and “What kind of results?” didn’t help to clarify things either. So I learned to actually draft a little mission statement, like “Provide as much information as I can to management about issues I believe will cause lost clients or calls to tech support.” Sometimes someone would get very upset and change my mission statement, but that was fine by me because then I knew what their expectations were.

The intent question is what helped me stay focused on what really mattered.  The fact is that if the business we are working for fails, it doesn’t really matter how well tested a piece of software is.  No business pays testers for the sake of paying testers.  The truth is that most organizations feel that testers are an unfortunate necessity.  That truth makes it all the more important that our work add at least equal value to the business, as it costs the business to have us there in the first place.

It is at least as important as it is to have the situation, mission, and intent information individually, to look at those three items collectively. Often when I look at these three items collectively, what I find is that those three items contradict or work against one another. That is typically a good indication that either I have gotten things very confused, or it’s time for some stakeholders to have a meeting to figure out what is *really* important.

uTest: To our knowledge, no one has produced a movie entirely about software testers. However, there have been some notable software bugs in motion pictures (HAL from 2001: A Space Odyssey, the “Doomsday Machine” from Dr. Strangelove, the compound interest bug from Office Space, et al). If you had to suggest a movie that would inspire testers, what would it be and why? There are no wrong answers, except for The Net starring Sandra Bullock and Rocky 5.

SB: WarGames, 1983. All I have to say, is if you’ve built a computer capable of running WWIII simulations, and that is the same computer that controls the launch commands for nuclear warheads, and that computer is accessible via brute force hack of dialing phone numbers over a modem, you either didn’t have any testers or you didn’t listen to them.

Editor’s note: Here’s part II of the interview.