Testing the Limits with Michael Bolton – Part III

In the third and final part of the Michael Bolton trilogy, we cover advice for new testers, his hypothetical banishment from Software Land, the blogs he reads and more. Did you miss our earlier interviews? Here’s Part I and Part II.

uTest: Hypothetical: You’ve been banished from testing – nay, ALL software-related activities – for the rest of your days. What will you to earn a living?  What hobbies would you pick up to fill the intellectual void?

MB: Who knows?  For fun, I’d keep playing mandolin, probably. Teach, maybe. Write. I’ve worked in theatre stage management, been a book-keeper, tended bar, worked in a comedy club. In high school I worked in mail rooms during the summer. Whatever I’ve picked up in life, it was because something needed to be done and I was there to do it.  If it didn’t seem like much at first, I started to learn about it quickly. When you invest a little bit of effort to figure out your job, you learn how to makes it faster and better and more interesting. It turns into this great feedback loop. Any job can be more fun when you set out to master it.

uTest: Tell our testing community something about you that your most avid readers don’t know.

MB: While walking through the woods on an island near Vancouver recently, I found myself being quiet and brief, which I like from time to time. Practically nobody knows that.

Lots of people probably don’t know how much I’m eager to help people out. All of my work—courses, articles, conference presentations, this interview—comes with lifetime free technical support. Have a question? Just ask. I might not answer right away—supporting the family with paying work takes precedence over supporting the community—but I’ve never knowingly turned anybody down, so if I don’t answer right away, be persistent. James Bach makes the same offer, by the way. We’ve found that it’s a great way not only to help people, but also to explore problems and come up with solutions and learn things that can help our clients.

uTest: If you were talking to a newbie tester, what advice would you give him or her to set their professional journey off on the right foot?  How about for a 10-year veteran tester?

Continue Reading

Testing the Limits with Michael Bolton – Part II

In the first part of our interview with Michael Bolton, we grilled him on the emergence of the Weekend Testers, sensible metrics, Michael Bolton the pop star and a bunch of other topics. In part “deux” of our interview, we tackle the necessity of tester passion, how emotions affect testing, and the greatest threats to the profession. Check back tomorrow for the final segment.

uTest: There’s a lot of passion amongst testing thought leaders about the best way to test, or the best way to manage or train testers.  Often that passion overflows into heated debates.  How can this passion best be channeled to improve the state of testing?

MB: First of all, we should welcome debate, and we should welcome skilled argumentation as part of the art of construction and practice of persuasion. I’ve found, though, that it helps to remember that we’re exploring and challenging ideas. That means it’s good not to get too personally invested in certain ideas, because we’re always learning more, and because changes in context can mean big changes in what needs to be done.

That said, there are some ideas that seem robust for me. I believe that it’s unethical to dumb down people or the work that they do. I believe that we should focus our craft on learning, and learning how to learn rapidly. How can we improve the state of testing? By recognizing that software development is a web of people who are related in service to each other. That means putting people and social issues first. Get that right, and everything else will follow.

Suggestions are cool.  Standards are something else.  No group should be dictating to other people how they must test unless there are compelling human health and safety reasons for it. Do you really believe that the standards people know anything useful about your business? That the force of government-supported regulation, created by busybodies, should weigh on how you do your daily work? And if your answer is No, what are you going to do to get it stopped?

Continue Reading

Testing the Limits with Michael Bolton: Part I

We’re thrilled to have Michael Bolton as the latest victim of our Testing the Limits series. As the founder of DevelopSense, Michael has traveled the world teaching the craft of software testing to businesses and individuals alike. Since 2005, he has specialized in courses on Rapid Software Testing – which he co-authored with James Bach. Michael is also a prolific writer, and his publications include hundreds of articles, essays and columns. Aside from his blog, you can keep tabs on his latest work through Twitter.

In Part I of the “trilogy” we discuss the Weekend Testers, testing abroad, how numbers can enslave managers, and of course, his pop-star namesake.

uTest: You’ve been a thought leader in the testing space for a while now, but people still seem to get you confused with Michael Bolton (the singer) on Twitter. Ever thought about creating a tester alias? Or have you considered asking him to change his name since “he’s the one that sucks.” Assuming you (and our readers) have seen Office Space, I bet this joke never gets old.

MB: Yeah, it never gets old.  Try renting a car with this name.

A couple of things on that. First, Office Space captures very well what it’s like to have my name. Second, it’s not his real name; he changed it already. Way back when, before Office Space, I was working in tech support at Quarterdeck Canada.  American callers would occasionally turn north when there were long phone queues in Santa Monica. On one call, when I introduced myself to the customer, he laughed. “Really? That’s your real name?” “Yes, really,” I said, expecting one of the usual jokes. He said, “You know, it isn’t his real name. I used to be his bass player.”  The singer’s real name is Bolotin, but according to the bass player, there was no hope that radio DJs would ever pronounce “Bolotin” right, so he changed it.

uTest: We recently interviewed your friend and colleague James Bach, who had high praise for a group called the Weekend Testers. Can you give our readers a quick recap of what this group does, and whether or not you’re on board with their testing philosophy?

Continue Reading

Testing The Limits — 2009’s Top Posts

Testing The LimitsAfter we re-launched our brand in May, we decided that the uTest blog needed to be more than just uTest employees talking about uTest events, uTest awards and the uTest community (see how repetitive that gets?).

Writing witty, thought-provoking content is really hard.  And we’re pretty lazy, but fortunately we know some extremely smart & funny people.  So we invented the Testing The Limits series, in which we interview leaders from the worlds of testing, software, entrepreneurship and crowdsourcing.

We’re immensely grateful to these talented, busy people, and we have much more planned for the Testing The Limits series in 2010.  But before we flip the calendar, these posts from this year are worth another look:

June: James Whittaker – Author, Professor and Testing Evangelist at Google

July: Rosie Sherry — Founder of the UK-based Software Testing Club

August: Andrew Muns — President of Software Test & Performance

September: Jack Margo — SVP of Internet Operations of Developer Shed

October: Jon Winsor — Author, Crowdsourcing Expert, and Founder of Victors & Spoils

November: Matt Heusser — Software Testing Author, Professor and Testing Manager

December: James Bach — Software Testing Author, Teacher and Speaker

We have some great guests and ideas lined up for 2010, including software execs, QA thought leaders, and famous journalists & authors.  As always, the goal of Testing The Limits will be to inform, to entertain, and above all else, to help our readers get to know these thought leaders who are worth following and listening to.

Have a suggestion for a future Testing The Limits guest?  Drop us a note or tell us in the comments section.

Testing the Limits with James Bach (part 2)

Yesterday we posted Part 1 of our interview with James Bach, where he discussed tester certifications, faking test projects, his latest book and wide range of other topics (including life as a freelance sentry in a parallel universe). Today, for Part 2, we discuss tips for automated checking, what makes a good tester a great tester, his flying lessons and much more. Enjoy!

uTest:  Do you see the quality of resources in the testing field increasing or decreasing (tools, training, certs, et al)?  What do you think are some of the drivers of that change?

JB: There are many good resources out there, and yes there are resources getting better. There’s testingeducation.org and the Weekend Testers project, to name two. At the same time there are terrible things out there (such as certification and all the stupidity that goes with that). You have to be a smart consumer, because it seems to me that the bad stuff has always outweighed the good stuff by an order of magnitude or so. Maybe by two orders of magnitude.

uTest: When it comes to automated checking, what are some of the key opportunities to employ it that generally generate a positive ROI? Are there any good rules of thumb that can be used, i.e. if you plan on executing the same test 7 times, then it is a candidate (understanding of course that some assumptions need to be made to answer this)?

JB: Here’s how I think of it:

– Is the product highly controllable and observable? A command line tool that provides its output solely to the console window is inexpensive to automate, compared to an iPod touchscreen app. I want to get under the GUI.

– How expensive is the tool I’m using? I urge you not to use expensive tools, even if they work. Never let your manager buy them. Because expensive tools become something you MUST use, even if they don’t work. A free tool may be freely abandoned. This gives you flexibility.

– How well can I automate the oracle? Will the bugs be able to elude my automation because it can’t tell if a complex graphic is rendered correctly?

– What is the learning and testing value I’m giving up by using automated checks? I find that doing a test multiple times also causes me to learn and see new things in the product. Furthermore, when I re-run tests, I often run them in a different way, and that allows me to find new bugs.

– Can the automated check be parameterized and randomized, so that I get lots of similar checks for very little additional investment? I like automation more for data intensive testing, because I get new tests just by changing the database.

– Is the technology “Pyramid shaped?” In some products lot of underlying code boils up to one simple output, by placing checks on that output, we may be able to find lots of bugs. In other products, there are many different pathways, and you need a lot more checks to get decent coverage.

– How critical are the checks to the business? Is this critical functionality? Is it a common usage scenario? There are candidates for smoke testing.

– Is this part of the product especially prone to breaking? If so, that may be good for automation, UNLESS, it breaks in a way that breaks the automation.

– When I automate, I do it incrementally, in small bits.

I want automated checks for high value, highly testable parts of the product, and I want to do them in such a way as they aren’t constantly breaking or giving me false readings. I want to augment those checks periodic sapient testing as a cross-check.

uTest:  What characteristics and practices make for a good tester?  How about a great tester?

Continue Reading

Testing the Limits with Matt Heusser (part 2)

Today, we finish up our “Testing The Limits” interview with Matt Heusser.  Be sure to check out part 1 of this interview.  In this installation, we’ll discuss which mobile app testing, his OS of choice, and the testing blogs and sites Matt reads.

What other blogs, sites or message boards do you read to stay on the leading edge of all things testing?

MH: I’ve got a bit of a bias there, as I write a monthly column for Software Test and Performance Magazine. You can get the PDF version for free every month. I also subscribe to Better Software Magazine. For communities, I like softwaretestingclub.com and the agile-testing discussion list. For blogs, well, there’s James Bach, Cem Kaner, and Michael Bolton. Adam Goucher has been doing a lot of writing lately, including editing the new book, Beautiful Testing, which I contributed a chapter to. And I spent a fair amount of time working on my own blog, “Testing At The Edge of Chaos.”

Recently I’ve been getting to know people by working on projects with them; my friend Chris McMahon started a Google Group on Writing About Testing which I found to be a blast. Through that group (and the Miagi-Do School) I’ve met quite a few new bloggers: Markus Gaertner, Lanette Creamer, Marlena Compton, and Ajay Balamurugadas come immediately to mind.

What OS are you running right now? What’s your browser of choice? Anti-virus? Inquiring minds would like to know.

MH: Max OS X, and I have to support all of them, so I run all browsers. Anti-Virus? Dude, I told you, I use a mac, and I try to avoid the questionable websites that host the viruses. What have you been browsing lately, Mr. Johnston?  (Ed. Note: I’ll ask the questions around here.)

Continue Reading

Testing The Limits With Matt Heusser (part 1)

matt-heusserIn this month’s installment of “Testing The Limits”, we sit down with Matt Heusser (@mheusser) — prolific blogger for STPCollaborative, thought leader and testing extraordinaire.  We’ll discuss the state of software testing, SpeedGeeking, the role of chaos in testing software, and the lack of fistfights at STPCon 2009

uTest:  We loved the SpeedGeeking session you led at STPCon, so we’re going to flip it on you – If you had just five minutes to teach, motivate or inspire the uTest audience about software testing, what would you say?
MH: Well, I’d start by asking the audience what they are doing today – what’s the greatest point or opportunity they feel – and asking what options they see to improve. Most of the time, I hear that testing is “too slow” or “the bottleneck” or something like that.

So I suggest taking two weeks and actually measuring how the team is spending its time. Oh, not for reporting – it is very important the team stop the time tracking after two weeks and not hand individual metrics into management for evaluation. Instead, we want to use the numbers for improvement. For example, many of the people I talk to can spend 80% of their time or more in meetings, working on documentation, working on compliance activities, doing email, and so on. That only leaves 20% of the time to test! Just pushing those numbers from 80/20 to 60/40 will double the amount of time the team spends actually doing testing.

Another thing to look at is the amount of time spent trying to reproduce defects, document defects, file bug reports, “verify” fixes, and so on. We think of these activities as testing, and they can take a substantial chunk of that 20% – but they are really accidental. That’s not a testing bottleneck – it is a development bottleneck. If test can work with development to improve the quality of the software prior to code complete, that will improve the speed of the whole system. Realizing this, and having a little bit of data to “prove” it, can help the entire system improve.

So if I had five minutes, I would say start with measuring how you track your time, and ask yourself if this is the best use of your time and what can change. Sometimes, the big boss will say “no, we absolutely need you to fill out all seven pages of documentation per test run”, and you can say “ok.”  Six months from now, when someone asks why the big project is late, you can point out that the business made an explicit decision to pay the full price of defined process. You presented options and those were not accepted.

That won’t save this project — but it might save the next.  It also turns out that actually testing tends to be much more fulfilling than documentation and compliance activities. Who could have guessed?

Lots of contrasting opinions at last month’s STP Conference. While there were no fist fights (that we heard about anyway), what did you see as the most contentious issue? And where do you fall on this issue?

Continue Reading

Testing the Limits with John Winsor

Having grilled some of the top minds in the software business, this installment of Testing the Limits will deviate johnwinsorslightly from the norm. With us this month is John Winsor – author, entrepreneur and crowdsourcing expert.

After a successful career as a journalist and magazine publisher, John founded Radar Communications in 1998, where he implemented a variety of academic-based market intelligence tools to help some of the country’s most progressive companies learn from key voices in their communities. Today, he offers that same advice as the VP/Executive Director of Strategy and Product Innovation at Crispin, Porter + Bogusky.

John has written extensively on the subject of crowdsourcing, having published the popular 2005 book Spark: Be more Innovative through Co-Creation. With his latest book Baked In: Creating Products and Businesses That Market Themselves now hitting the shelves, John was kind enough to sit down with us to discuss the future of crowdsourcing, the premise of his new book, and the best (or worst) rock-climbing movies of all-time.

uTest: The hottest debate in crowdsourcing right now is the “fall” of traditional advertising or design firms and the “rise” of crowdsourced services. In your opinion, what does the future of crowdsourcing look like? Is the world ready for what you call the “digital tsunami?”

JW: Well the future of crowdsourcing is definitely bright, but there are still a lot of unanswered questions in people’s minds. Those who are skeptical of crowdsourcing question its ability to truly connect people. With crowdsourcing, you no longer have all of these professionals working together in the same building – that alone is often too much for some people to come to terms with. The idea of a crowd aggregating to solve business problems in a virtual environment is entirely new to most people, even though the underlying trend has been developing for years. The difference now is that it simply can’t be ignored.

uTest: So you see crowdsourcing as eventually obtaining mainstream acceptance?

JW: Absolutely. People are starting to see the full potential of this model, especially on the client side of the equation. There was a time when most people viewed crowdsourcing as chaos – like the inmates running the asylum, and that’s no longer the case for a growing number of people. So I think we’re just getting started.

Continue Reading

Testing the Limits with James Whittaker (part one)

Once a month, we’re going to “test the limits”, interviewing a leading thinker in the world of testing and quality.  It james_whittakercould be a journalist, an industry analyst or an exec from a top software company.  To kick this program off, we could think of no better person than our good friend, Dr. James Whittaker.  So we recently interviewed James by bouncing emails back & forth over the course of a few days.

Several of these questions came directly from our community of testers.  The whole exchange is fairly lengthy, so we’re splitting it into two posts.  Come back and check out the 2nd half later this week.

uTest:  So the news is out about your move to Google. What prompted you to make this move?

JW:  I didn’t so much leave Microsoft and I did join Google. I was attracted by all the Googlers I met at conferences and what I read on their blogs about the way they test. When they offered me the opportunity to be a part of it, one might even argue an important part of it, I found it impossible to decline.

uTest:  Is there something about Microsoft you’ll miss the most?

JW:  Yes, the breadth of both products and expertise. You literally have every type of software imaginable and a chance to collaborate with the people who make that software. From an intellectual standpoint, Microsoft is mind-blowing.

uTest:  What specific work at Microsoft did you enjoy the most?

Continue Reading