Software Testing Blog

In the third and final part of our Q&A with testing guru Scott Barber, we discuss what qualities to look for when hiring testers; a world without testing teams; when to start blogging, and much more. If you missed our earlier interviews, here’s Part I and Part II.

uTest: It looks like you’ve hired your fair share of testers – what skills, experience and characteristics do you look for when making these critical hiring decisions?

SB: The most important things I look for in an individual are aptitude to learn whatever technologies, business information, processes, etc. that they will be expected to test – that and investigative curiosity.  Beyond that, I’m always looking to build a well-rounded and diverse testing team. I find that a team full of testers “like me” leads to us all missing the same kinds of bugs, so I’m always looking for a breadth of skills and experiences across the team. I’ve hired aircraft mechanics, medical billing specialists, call-center representatives, technical sales representatives, subject matter experts, developers… oh yeah, and even a few testers.

I know how to teach people technologies, the basics of a new programming language, how to report bugs, how to document process, and how to generate test ideas from whatever information we have.  What I don’t know how to teach people is how to be curious, the desire to investigate their curiosity, or to be excited by the thought of investigating things that strike them as odd.

So I guess I’m always on the lookout for someone with a tester’s attitude who fills a gap in perspective of the current team whose own knowledge or skill gaps fall in areas I know how to teach them or get them trained in.

uTest: Hypothetical: All testers are barred from practicing their craft for an entire year. What does the software industry – nay, the modern world – look like without their services? Please be as dramatic and hyperbolic as possible.

SB: Sadly, I think most of the world wouldn’t even notice. I think for the period of a year, non-testers would pick up enough of the slack to keep things moving, and enough individuals would perform enough “heroics” to keep the software world running. Sure, more bugs would make it into production, more software would ship late, and there would be more evening news reports of software failures, but I’m not sure the general public would even notice.

In fact, if software producers were smart, they’d pass some of the savings they were realizing by not having to pay testers on to the general public, making them even less likely to mind a few extra bugs.

But eventually, some kind of tipping point would be reached and the public would demand less buggy software. That is when the disaster would actually happen.  By the time companies re-engaged with testers, re-integrated them into their new (clearly, more streamlined, process), started getting results, realizing just how much re-work needed to be done and how long it would take to get software back up to the “old” standard, the public would be so furious that the “old” standard would no longer suffice.  Of course, achieving the new, higher, quality standard would take more time, and make the software cost more, and thus make the public demand yet *more* for their money.

Interestingly, if the software industry managed to survive for long enough to find a balance point at the tail of this downward, then upward spiral in software quality, I think it would be the best thing that could happen for testers. The publicity would lead to higher quality training for testers, more education about testing for developers and managers in school and on the job, better hiring methods and practices, and more respect for testing as a career. As the public became aware that software doesn’t have to be bug-infested (sure, all software will always have some bugs, but a couple of lady bugs are far more acceptable than a cockroach farm) more buyers will become more willing to spend a little extra for better quality, leading to more jobs for skilled testers, leading to more people wanting to become skilled testers, which would increase enrollment into education programs, leading to more research and development, and on, and on.

Maybe this isn’t exactly how it would go, but I do believe that it would take far too long for the public to notice and far too long for companies to start losing more money than they save by not paying testers. I also believe that unless something like this does happen, the public may never realize that they have been conditioned to accept far lower quality from software than they do from non-software items of equivalent cost. Think about it, how many times in the last year has a software bug ruined your day?  If the building materials used to construct your home were of the same quality of the software you use, how would you feel about your home?  What would you think if you had to patch a wall every time you patched your software?

Testers can’t solve the quality problem on our own. Higher quality has to be driven by conscientious business owners or disgruntled buyers. The best testers can do is try to make those executives as aware as we can of those bugs we can find while operating with not enough training, not enough time, not enough people, not enough equipment, and not enough support.

uTest: Are great testers born or built?  And if they’re built, then how the heck does that happen?  Share a little wisdom with the newbie testers and college kids who will join the ranks in the next year.

SB: I think that the potential to be a great tester is a part of a person’s personality that more or less exists or doesn’t by the time that person reaches college or the job market, but I don’t think that potential alone is enough to make someone a truly great tester.  Becoming a great tester is like becoming a great anything-else. It takes some natural ability, some opportunity, some good coaching, some relevant experiences to draw from, a lot of self-motivation, and a lot of practice.

Every now and again, someone might have so much of some of these items that they can get away with less of the others, but that is the exception. If you want to be a great tester, just like if you want to be a great musician, writer, or athlete, practice all you can, find a good coach or mentor, learn everything you can about technologies, designing experiments, systems, psychology, human interfaces, anything you can imagine being related to what you might end up testing or might help you better understand what might matter to the people who use what you might end up testing.

When you think you’re getting really good at something, start blogging it, then teaching it to others.  Don’t be discouraged when you find that it doesn’t really work for them at first.  Keep refining and revising until you believe those you are teaching are at least as good as you thought you were when you first started trying to teach them. When your students reach that level, pick the thing you are next best at and do the same thing.  You’ll know you’ve become a “great tester” when your current students have been referred by your former students, and your former students are trying to hire you instead of just trying to pick your brain.

uTest: [In the voice of deep, gravel-voiced movie preview guy] The year is 2020.  And in a world gone mad… where only one man knows what really happened… only one man can face the danger, beat the odds and reveal the truth… (insert pyrotechnic explosion here).  Okay, I don’t know precisely where I’m going with that, but what do you think testing look like in 2020?  What needs to change ASAP in the world of software design, development or testing in order to ensure a future of safe, reliable apps?

SB: The first thing that needs to happen is that testers, developers, managers, and testers need to learn what a “test” is.  Sounds silly, right?  It’s not.  Almost every day I encounter someone who wants to know “What test can I conduct to prove that…?”  While the answer (“There is no single test to prove that…”) it demonstrates a fundamental misunderstanding of what a test is.

Think about all the tests you have taken during your days as a student.  Do you believe that any of them proved that you had learned what the instructor had intended?  How about your driver’s test?  Do you believe that passing your driver’s test proved that you were going to be a good driver?  What about the last time you or someone you knew had a blood test?  Did you believe that blood test was going to prove the person was healthy?  Be honest now.

Of course you didn’t.  But somehow, most folks seem to think that software tests can prove that something will work in production.  The best we’re ever going to be able to do is demonstrate ways in which the software already isn’t working, demonstrate that the software doesn’t currently meet some measure of “goodness”, demonstrate that it is possible that the software may work in production, and/or collect information of potential value to someone who matters.  Even that depends on our tests being reliable and accurate.

Before testing can truly advance, folks need to recognize that testing is an application of the scientific method, that testing involves experimental design, and that tests don’t pass or fail, they provide useful information or they don’t.

Most of what happens today under the label of testing is nothing more than simple validation of expectations.  Testing is a search for information.  Testing is most valuable when expectations are not met.

Don’t get me wrong.  Someone had better be validating that expectations are being met or no one is going to want the software we’re developing, no matter how much we test it, or how few bugs are in it.  But we can’t stop there.  Technologies evolve too fast, users do things the most creative of us would never imagine before our software is retired for us to think that validation in some non-production environment is enough.

10 years ago, I heard a lot of managers and executives saying “What do you need testers for?  If you’d just develop it right in the first place, they’d have nothing to find.”  I don’t hear that very often anymore.  Today, executives understand that is an unreasonable sentiment.  Now they seem to think that testing can prove and/or control quality.  Maybe, 10 years from now, they will have figured out how to get the most value out of testing by asking them to actually test for the purposes of helping developers build better software and helping executives make better decisions.  I hope that is the case, but if there isn’t a change in the way “a test” is perceived, I don’t think it’s very likely.

uTest: We’ve been fortunate enough to interview the likes of Whittaker, Heusser, Bach, Bolton and Bach in the past for our ‘Testing The Limits’ series… who should we interview next?  Give us a few names who would continue this great (albeit young) tradition and blow our minds?

SB:

Rob Sabourin
Mike Kelly
Cem Kaner
Elizabeth Hendrickson

Editor’s Note: If you have a suggestion on who should be our next guest for Testing the Limits, let us know.