The amount of testing knowledge added to the web each day is nothing short of amazing. Unfortunately, the scores of new techniques and ideas can be a bit overwhelming, which is why it’s helpful to revisit the classics once and awhile. And that’s what today’s post is all about. Actually, it’s about risk-based testing, but you get the idea.
Way back in 1999, testing guru (and frequent guest of the uTest blog) James Bach published an essential guide on the topic of risk-based testing, which you can find here (PDF). I’ve extracted a few excerpts that will be a reminder for some, an introduction for others, but extremely valuable for anybody that continues reading.
First off, here’s his definition of how to perform risk-based testing:
- Make a prioritized list of risks.
- Perform testing that explores each risk.
- As risks evaporate and new ones emerge, adjust your test effort to stay focused on the current crop.
And here are some testing heuristics he mentions as part of the “outside-in” approach:
- Capability. Can it perform the required functions?
- Reliability. Will it work well and resist failure in all required situations?
- Usability. How easy is it for a real user to use the product?
- Performance. How speedy and responsive is it?
- Installability. How easily can it be installed onto its target platform?
- Compatibility. How well does it work with external components & configurations?
- Supportability. How economical will it be to provide support to users of the product?
- Testability. How effectively can the product be tested?
- Maintainability. How economical will it be to build, fix or enhance the product?
- Portability. How economical will it be to port or reuse the technology elsewhere?
- Localizability. How economical will it be to publish the product in another language?
Lastly, here is James’ summary of the importance – and practicality – of risk-based testing:
If there’s a magic to risk-based testing, it’s the magic of noticing the signs and clues, all around you, about where the problems lie. Some people do this without consciously thinking about it, and maybe that’s good enough. But when a problem slips by you because you couldn’t do perfectly exhaustive testing, you may be called upon to explain why you did what you did. Management may assume that you did a sloppy job, and they may not be impressed with the standard argument that all testing is incomplete. That’s when it’s nice to have that risk list or matrix. With risk-based testing, you can show management that you strive to make the best use of the resources they invest. They’ll respect you for that.
What are some other required readings on the subject of risk-based testing? Leave your suggestions in the comment section below.