Bug Reporting Lessons From Toyota: Are Your Brakes Show Stoppers?
In light of Toyota’s recent quality issues, the number of formal consumer complaints has risen above the norm. To make matters worse, Toyota has had an extremely difficult time making sense of all this new feedback.
Why? Well, if you are an experienced QA professional, you know exactly why.
A recent article about how to write a useful NHTSA (National Highway Traffic Safety) complaint should strike a chord with software testers. The complaint template is very similar to the bug reports we all know and love. In fact, they both serve the same purpose: defect reporting.
Consumers can learn a few lessons from software testers – and vice versa – by taking a look at some key excerpts from the article:
Include data that will help the manufacturer better understand the problem:
- Facts about your vehicle and maintenance records
- What you did and how the vehicle responded
- Evidence and extra details
Exclude information that does not help — at all:
- Multiple problems in the same complaint
- Your feelings (anger, frustration, nervousness, etc.)
- Spelling errors
If you follow the uTest blog, you may have come across a well-written post by guest blogger and uTester Joseph Ours: Respect the Defect: Advice that will change the perception of testing. The philosophy of proper defect reporting is well described by Ours, and may be a great primer for consumers filing NHTSA complaints (though we hope you’ll never have to).
Of course, experienced software testers are sometimes required to use screen shots and video capturing tools to recreate the defect. Interestingly, not all of the Toyota “bugs” have been reproducible, including one highly publicized incident. This makes accurate defect reporting even more important to developers and engineers, who must address the root cause of the issue as quickly as possible.
So instead of trying to recreate that floor mat issue you had on the way to the grocery store, we suggest that you simply file your complaint in a clear, concise and calm manner. That goes for software testers as well.
Why not multiple problems in the same complaint??? After all, I have a forum to list all my complaints and would prefer 1 person handling it…. or having to track 1 thread wiht 10 complaints, rather than 10 threads with 1 complaint…..
I’ll relate this to a software development/testing perspective by sympathizing with our developer/programmer friends. The ones that I’ve worked with do not like multiple issues in one bug report. When they tackle a known bugs list, they like to attack bugs one at a time and be able to group similar bugs together. Lumping multiple bugs together in one report prevents them from doing so.
From a product manager’s prospective, it is also easier to keep bug reports at the single entity level for reasons such as reporting, comparing bug counts with previous builds, etc.
What do others have to say about this topic?
I disagree about excluding feelings from bug reports.
In software bugs reported by testers or customers (differently than the officially dry NHTSA complaint), feelings are important.
More than being about a technical problem, bugs are about feelings: Anger, frustration, impatience, even despair. Describing feelings when summarizing the bug helps others to understand the real value that the bug is detracting from.
That is not to say that the report should be written from angry or nervousness. Write it calm and objectively, but don’t forget your customers’ feelings – you are the one responsible for defending them.
I agree to the fact that the issues and complaints should be uniquely logged / reported because the addressal/ solution / closure of the same becomes much easier and simpler.
At the same time, I understand the reason behind the fact that the users should have a parameter to provide their feelings which would help the technicians and managers prioritize the problems at hand considering the user experience along with the technical criticality of the issue.
It can be something of a grading system (like the ones we use in IT industry):
1- Showstopper
2- Critical
3- High
4- Low
I’m firmly in the “one bug per report” camp. Things are more efficient and easier to process when data is clean and structured.
And if a bug report is going to be classified by severity, by type, and/or by frequency, then it’s necessary to create a 1:1 relationship between bugs and reports.