Bolton and Bach on Testing vs. Checking

checkingWhat if most of the negative stereotypes associated with software testing were actually those of software checking? In other words, what if most people’s view of testing was completely wrong? Before one could answer those questions, they would first have to know the difference is between testing and checking.

Michael Bolton and James Bach have an idea or two on the difference between testing and checking. In fact, the two co-authored a post on that very subject for the Satisfice blog. I highly encourage you to read their post in its entirety, but I did want to briefly highlight their formal explanation, as I think it will be a subject we come back to in the near future. Here is how they define it:

Testing is the process of evaluating a product by learning about it through experimentation, which includes to some degree: questioning, study, modeling, observation and inference.

Checking is the process of making evaluations by applying algorithmic decision rules to specific observations of a product.

And then, to give a clearer picture, they offer some key distinctions/assumptions:

  • Testing encompasses checking, whereas checking cannot encompass testing.
  • Checking is a process that can, in principle be performed by a tool instead of a human, whereas testing can only be supported by tools. Nevertheless, tools can be used for much more than checking.
  • Testing is an open-ended investigation– think “Sherlock Holmes”– whereas checking is short for “fact checking” and focuses on specific facts and rules related to those facts.
  • Checking is not the same as confirming. Checks are often used in a confirmatory way (most typically during regression testing), but we can also imagine them used for disconfirmation or for speculative exploration (i.e. a set of automatically generated checks that randomly stomp through a vast space, looking for anything different).
  • One common problem in our industry is that checking is confused with testing. Our purpose here is to reduce that confusion.
  • A check is describable; a test might not be (that’s because, unlike a check, a test involves tacit knowledge).
  • An assertion, in the Computer Science sense, is a kind of check. But not all checks are assertions, and even in the case of assertions, there may be code before the assertion which is part of the check, but not part of the assertion.
  • These definitions are not moral judgments. We’re not saying that checking is an inherently bad thing to do. On the contrary, checking may be very important to do. We are asserting that for checking to be considered good, it must happen in the context of a competent testing process. Checking is a tactic of testing.

As I alluded to in the beginning, I find it interesting to note that many of the negative stereotypes of testing actually apply to what Bolton and Bach define as checking, for instance:

  • Testing is a not a boring task. Checking, on the other hand, very well could be.
  • Testing is not easy. Checking, on the other hand, is sometimes quite easy.
  • Testing cannot really be done by computers alone. Checking can (and probably should) be automated.
  • Human testers will never be obsolete, but human checkers (for lack of a better word) might be.

Do you have your own definitions of testing and checking? If so, please share them in the comment section below.

Until next time, happy testing (happy checking, too).

Essential Guide to Mobile App Testing

Comments

  1. Pradeep Lingan says

    To me Checking and Testing are same. It is up-to the people to see(take) it in different ways.

  2. Alex says

    Did you heard the words like ‘Check post’ or ‘Ticket Checker’ but not ‘Test post’ or ‘Ticket Tester’? For me, anything with ‘digital’ answer like yes/no pass/fail is a check which is more visible to end user and the process of performing this check is a test!

  3. Peter Russell says

    The idea of “testing” as described in the article is just another example of “sub management” arrogance. When IT management is weak, as occurs often, analysts, developers and testers all try to be the best busy bees they can be and second guess missing process and deliverables. When the emperor has no clothes, you still have to work and get paid, when the emperor
    (poor management) is untouchable you invent hyperbole to make yourself feel better about coping with project failure.

    Trying to define a core moral or technical argument behind something as silly as checking or testing is to be part of the problem in a wider sense. On the other hand software testing as either science or art is in a lot of pain, the community that so strongly identifies as testers is in a lot of pain. Does this kind of pseudo technical distinction help testers cope by bolstering the idea that they can still make a difference in a cruel world that continues to undervalue them … maybe.

    Michael and James, please keep writing, These are fundamental issues with the state of the craft and its supporting subculture. Well done.

  4. Kathleen Bowers-Pinette says

    Checking, to me, is just making sure that the basic functionality of the application works as expected. Click this button, it should do “THIS”. I agree that these types of checks should be automated and as a tester, this type of “testing” is VERY BORING! Testing is allowing me as a tester to use my imagination to explore the whole application, not just the modules that comprise the application. I have seen time after time, that just using “Checking” does not expose some of more serious defects in the application. Checking is necessary, however, I my opinion, if we are not “testing” the application, we are shipping a product that has not been fully investigated.

  5. says

    Totally agree. Checking is not the same a judging the software for usability and sensibility.

    Example – Prometheus (the alien movie) –
    The QA CHECKED to see if every visual in the frames, the flawlessness of the costumes and sets, the top notch acting, the breathtaking cinematography and so on.

    The QA did not TEST the script to see, if made sense, probably because they were not allowed to do it, just like a typical IT company.
    Srilu P

Leave a Reply

Your email address will not be published. Required fields are marked *