Software Testing Blog

Our Testing the Limits guest this month is Ben Simo. Known as the “Quality Frog” on Twitter, Ben is one of the most insightful and entertaining testers in the business. A proponent of the context-driven school, Ben has more than 19 years of experience testing software and developing testing tools. He currently lives in Colorado with his wife, two children, two dogs, five cats and fourteen – count ‘em – fourteen goldfish. For the full Ben Simo experience, go to his blog.

In part I of our interview, we get his thoughts on the Worst Bug Ever; his testing philosophy; what it means to be a defensive pessimist; testing certifications, the state of the industry and more. Be sure to check tomorrow for part II.

**************

uTest: Your “Is There a Problem Here?” series has been a big hit in the testing community. What’s the absolute worst bug that’s ever been submitted? And what can testers and developers learn from these type of mistakes?

Simo: Many of the bugs on IsThereAProblemHere.com could be argued to not be bugs. The software works or catches and reports an error condition; but in a way that it unnecessarily frustrates users. My hope is that people involved in creating and testing software can learn from these examples. Rather than only look for the obvious technical bugs, we need to be asking ourselves “Is there a problem here?”

We build software for the benefit of people. Software fails when it does something other than solve human problems.  Although not the worst items submitted, two items come to mind.

The first occurred on Christmas Day last year.  Twitter was full of complaints by people who received Sony’s new electronic book Reader device as Christmas gifts. The device worked except that Sony was not prepared for the Christmas Day rush on their servers as people attempted to install software and purchase books.  By not sufficiently preparing for the Christmas rush on their servers, Sony turned joy into frustration for many new customers. As a performance tester, I take this as a warning to seriously consider what events may cause a surge of demand for the systems I test.

The second problem that comes to mind is one I’ve repeatedly encountered with Blogger’s auto-save feature. I like features that help prevent users from losing their data.  While auto-save features usually indicate that software designers value their customers’ data, Blogger provides a great example of how auto-save can make things worse.  The Ctrl-Z undo option in users’ web browsers goes away after an auto-save occurs.  If a user fat-fingers text in a way that deletes content just before an auto-save occurs, there is no going back. An accidental Ctrl-A instead of a Ctrl-Z or Ctrl-X followed by another keystroke can permanently delete a document in an instant.

uTest: Gotta ask about the “Quality Frog” handle on Twitter. What’s the origin of this moniker?

Simo: A few people have told me “Quality Frog” looks like two random words from a Facebook captcha.

I’d like to be able to say that I carefully selected the name and that it signifies that I care about quality and I’m amphibious like a frog. I’d like to say something along the lines that I started life as a tadpole in the waters of programming and later grew legs to live on land and be a tester.  I could even say that as a Quality Frog, I now eat bugs for breakfast and help keep the waters clear. While such thoughts may have come to mind, the truth is that I came up with Quality Frog while pairing a variety of words with Quality in search of an available domain name. Frog came to mind as something that ate bugs and the domain name was available. Since then, I’ve continued to use it as a handle.

uTest: You’re a self-described “defensive pessimist”, which seem like good qualities for a tester to have. What other attributes come in handy in this line of work?

Simo: The term “defensive pessimist” comes from Dr. Julie Norem, a psychology professor at Wellesley College.  In her book, “The Positive Power of Negative Thinking”, Dr Norem describes defensive pessimists as people who typically perform worse when pressured to look on the bright side and be optimistic about things that concern them.  Rather than trying to think happy thoughts and only look at the positive, defensive pessimists imagine the worst case scenario; not to get depressed and become immobilized, but to develop solutions for what might go wrong in order to be better prepared. Defensive pessimists can make great testers, and are likely to annoy many optimists.

The third guiding principle of the Association for Software Testing states “AST views software testing as a cognitively complex activity that requires critical thinking, effective communication, and rapid self-directed learning.”  I fully agree with this.  Therefore, I find it essential that testers be critical thinkers, effective communicators, and self-educators.  Any one of these three things without the others will make us less effective as testers.

This doesn’t mean that every tester must master all three of these on their own.  My ideal test team would be comprised of people with a diversity of aptitudes, skills, and experience. I don’t want a team of clones.

uTest: Much like our previous Testing the Limits’ guests, you’re a critic of testing certifications. Yet some still see certifications as the only way to stand out from the “unskilled labor” crowd. Tell us a bit about why you’re a skeptic/critic of certs – and how they could be improved and made more relevant/useful/predictive.

Simo: I am not against certifications per se. I am against bad certifications.  I am against certifications that are presented to be something other than what they are. I am against certification bodies and trainers that prey upon people’s desire to stand out and tell people they can improve and certify their competence as testers with few days of training and a multiple-false test. In his keynote at the Conference of the Association for Software Testing (CAST) this year, Cem Kaner stated that If one can become certified in their profession in three days, they are a commodity, and don’t deserve much more pay than unskilled labor. I agree with Dr. Kaner. Rather than educate and help people stand out from the unskilled labor crowd, such certifications trivialize testing and encourage wrong thinking that testing is unskilled labor.  I want testers to be more than a commodity.

Many IT certifications, including testing certifications, are more about marketing than education.  These certifications are not good measures of skill, competency, professionalism, quality, or any of the many things those on the receiving end (of the marketing) care about. In my experience interviewing job candidates, tester certifications have not been an indicator of applicants’ testing abilities.

Software testing is a rich and diverse field. It is also a young field. Rather than feign maturity and simplicity where there is none, let’s embrace the diversity and youth. Let’s continue to learn. Let’s not lock in a set of context-free definitions and practices and make them a standard. Such standards will hurt the quality of software, not improve it.

Rather than pursue a certification, I encourage testers to get involved in a professional community. Find colleagues that challenge you and help you learn.  Seek real education that comes through interaction and doing over memorizing information useful in passing a multiple-false test. The Association for Software Testing offers a series of online software testing courses that facilitate deeper learning that you are likely to find in training focused on helping you pass a certification exam.

Editor’s note: We hope enjoyed part I of our interview with Ben Simo. Here’s part II.