Software Testing Blog

Our guest blogger this month is Joseph Ours, a recent Bug Battle winner with more than 12 years of IT experience, including software testing and quality assurance. In this post, Joseph advises testers to re-examine the way they report defects in software applications.

Testers and testing are viewed as a cost center in many organizations. If you look at the roles of other “main” players, you quickly see that testers face what I call an issue of intangibles. Here’s what I mean:

• Project managers – They are task masters driving a product to completion. Businesses absolutely want products created on time and under budget – which is why they are (correctly) viewed as an absolute necessity.

• Analysts – These guys get the great job of descriptively conceptualizing the idea. This is akin to a paper prototype, and gives the business the first real glimpse of how it might look and work.

• Developers – They are the cream of the crop. They get to create an actual product that businesses can see and feel.

• Testers – Well, we say if it works or not.

Everyone else gets viewed as making a positive contribution in getting a product to market. Unfortunately, testers are in the position of “finding” things that can mar the perception of what is being created. This can lead to confrontational situations that are, to say the least, unpleasant. So, how can we as an industry change this perception of our craft? There are many answers, to be sure, but  I would like to focus on a single element of our profession that elicits much of this ill will: defect reporting. Actually, I would rather call them observations, but that’s another post entirely.

The primary purpose of defects is to point out what we believe to be a notable observation about the application. Typically, a defect report is about something that doesn’t work the way we think it should.  This inevitably leads us to write a report where we layout the steps used to encounter the defect. However, one thing we often leave off of such reports is why we logged it. After all, a defect is an observation. As with all observations, there is a point of reference. For software, it can be a requirements document, UI standards, historical experience, etc. Without our reference point included in the defect, the receiving party is left to guess why we thought it was wrong in the first place.

This can lead to all sorts of issues, but it generally leads to at least one meeting for clarification. But the bigger impact is by NOT saying why we think it is wrong, as we can lose credibility and the opportunity to demonstrate tangible value.

Consider this: In addition to recreating the steps, what if we included a reference point as to why something is a defect AND articulated the loss of value to a stakeholder (usually an end-user)? Would others view our observations, and us, as more valuable?  Of course they would, and isn’t that the point?

For example, instead of saying:

“I entered in ‘John Smith’ in the name field, hit the tab key, and received a javascript alert”

We said:

“I entered in ‘John Smith’ in the name field, hit the tab key, and received a javascript alert. The javascript alert contained an error message regarding null objects. End users experiencing this behavior may be inclined to believe the registration process has failed and may leave the website.”

We have stated an observation, a point of reference, and a value item related to a stakeholder. But it is not just end-users that have a value concern. Let us take another example.

Instead of saying:

“I entered ‘<script>alert(‘me’);</script> into the name field and the application accepted it on submission. On the confirm registration page it displayed an alert message with the text ‘me’ and allowed me to successfully register”

We said:

“I entered ‘<script>alert(‘me’);</script> into the name field and the application accepted it on submission. On the confirm registration page it displayed an alert message with the text ‘me’ and allowed me to successfully register. The system allowed un-encoded fields to be submitted and subsequently executed commands. This could allow a malicious person to execute a cross-site script attack against the application; leaving our system and users vulnerable.”

Here we stated our observation, our reference, and our concern. This makes our observations more useful to stakeholders. Ultimately, this makes testers more valuable as well. We get the opportunity to demonstrate our value; for example, how our efforts improved end-user experience, the security of our system, and adherence to the conceptualized idea, to name a few.

This, of course, is precisely what we want. And one of the easiest, most effective ways to demonstrate the tangible benefits we testers bring to the table is to change the way think of – and write – defects in software.

Joseph Ours has more than twelve years of information technology experience in areas of requirements definition, analysis and design, program/ project management, full system life-cycle development, training, testing, and quality assurance. He has been a project manager, team lead, and team member, continually striving to provide the very best service and expertise in response to an ever-changing and increasingly complex IT environment. You can read his blog here.