You may be familiar with our Testing the Limits series where we interview thought leaders in software development and testing. These experts answer questions, give advice and share nuggets of wisdom that will help readers succeed in their careers.
How do you know they’re not just full of hot air? Well, for one, they probably wouldn’t be well respected members of the QA community if they were just lying all the time. But also, real life backs them up.
In this series, we’ll take excerpts from Testing the Limits interviews and pair them up with real life situations that prove these experts know what they’re talking about. There’s our first expert excerpt:
“Real testing, to me, should be based on investigating how the software allows people to deal with what we call ‘exceptions’ or ‘corner cases.’ That’s what we call them, but if we bothered to look, we’d find out that they were a lot more common than we realize; routine, even.” – Michael Bolton
This is what in-the-wild testing is all about. Ultimately, real life people will be using your software in real life situations. Sometimes, those real life situations are scenarios you couldn’t test for in the lab, didn’t occur to you or that you didn’t even dream of. But odds are your users will find them. While a crazy fringe use case will pop up from time to time no matter what you do, it’s important to take a step back, think through your testing and make sure you’ve come up with as many use cases as possible.
One of the best, most obvious and repetitive examples of “corner cases not really being that uncommon” can be found in video game development. Gamers are dedicated people and will try to make your software do things you wouldn’t believe. The biggest problems occur when they’re trying to do something reasonably within scope – uncommon maybe, but not unusual or unheard of. For instance, when Mass Effect 3 came out early last year the game encountered a bug that wouldn’t let players import customized avatars from earlier editions. If players set the appearance of their Commander Shepard avatar in the original Mass Effect and didn’t tweak it at all in Mass Effect 2, the character would revert to factory appearance when imported into game 3. From CNet: