Everyone wants to know what Apple’s going to say at their big press conference in a couple of hours. Will the iPhone 4 bugs prompt them to issue a recall? Will they send users a plastic case that supposedly solves the reception problems? Will they try to fix the defects with a software patch? Will they say they’re sorry and that this will never happen again? Will they tell NY Senator Chuck Schumer to suck an egg?
We’ll have to wait and find out. But here’s one thing they’re NOT likely to say (but they should): “We should have listened to our testers!”
[Update: See this TechCrunch story for a round up of the press conference]
One of the biggest pet peeves among testers and engineers (or anyone in involved in quality assurance of technology) is not being taken seriously when a serious issue is uncovered. For most companies, it’s generally a cross-site scripting vulnerability, an SQL injection or a browser compatibility flaw in the UI. For the iPhone 4, it was an antenna issue. As it turns out, many top executives – including Steve Jobs himself – were repeatedly warned about about the “death grip” well in advance of the product’s release. These warnings from respected internal resources were either ignored or not taken seriously. They should have listened to their testers.
But what should testers do when they find themselves in this situation? According to Bill Ricardi, they should report the bug and move on. A member of the uTest community, Bill gave his advice on this matter as part of our Guest Blogger series, writing:
You won’t always see eye to eye with the client. What you consider a critical bug, they might see as a non-issue (or worse, a ‘feature’). What you call a major security flaw, they might consider such a remote possibility that it doesn’t even deserve a mention.
You might ask how you bridge such a gap between your level of testing and the client’s level of acceptance and understanding of product integrity and the testing process in general. The answer is simple:
It isn’t your job to convert the client to your way of thinking. Yes, you can contest a bug that they reject out of hand if you were technically correct to report it. Sometimes they’ll accept it as valuable feedback, but most of the time they’ll just ignore any contested bugs. This is something that you have to live with.
You cannot impose your standards upon a client. No matter how passionate you are about the quality of your work and your understanding of their product, no matter how much you study the test parameters and the client’s requirements… only they know what they really want out of the test cycle. They WILL usually refuse bugs that they don’t consider ‘real’ bugs, because they’ve probably had meetings about what their programming philosophy should be, and they probably have a preconceived notion of what they have time and budget to fix.
So you’ll get hit with the dreaded duo of rejection reasons: ‘not a bug’ or ‘out of scope’.
What do you think? Should testers and engineers do more than raise their hands to highlight risks and defects? Asked differently, who is ultimately responsible for quality?
There are plenty of other testing and QA lessons to be learned here (first and foremost, the cost of discovering defects after a product is launched). Let’s see what Apple announces today, and then how the public and media respond to it. Something tells me this won’t be the last time we blog about this troublesome tale, and what it can teach us about launching high-quality products.