Software Testing Blog

In part II of our Testing the Limits interview with Lanette Creamer – aka “Testy Redhead” – we cover the need for Exploratory Testing; Matt Heusser and the “rebel alliance” of testing; how to create a popular testing blog; her stance on tester certifications and more from the wide world of QA. Catch up by reading part I, and when you’re done with this one, go check out part III.

uTest: In one of your recent blog posts, you mention Elisabeth Hendrickson’s STAREAST declaration that “Exploratory Testing Is Not Optional.” Why did this statement resonate so strongly for you? Do you think all testing managers should follow Hendrickson’s lead?

LC: As a frequent conference attendee and enthusiastic reader of testing blogs, I’ve seen many ideas about how to improve testing. I’ve been through countless industry trends, such as borrowing manufacturing ideas, extensive measuring schemes, and repeated attempts to automate all testing. The bottom line is exploratory testing works in practice. Not for a few months or years, but it works to find important bugs year after year no matter what other quality trends are happening. It works well side by side with automated checks, manufacturing ideas, and it can be used with session based test management to provide measures and metrics if needed. It is the one constant a tester can go back to and find bugs that impact the user experience. It is the meat in my testing sandwich. (My pun filled humor is the cheese.) To hear Elisabeth acknowledge the importance of exploratory testing in public shows me that agile testing is about more than just automation. Agile testing can be about a balanced approach to overall quality. It resonated with me strongly because it makes me hopeful for the future of testing on agile teams.

I think test managers should evangelize and defend what works well in practice on their teams. My experience has been that exploratory testing is generally undervalued considering how effective and practical it is.

uTest: What’s the deal with this “rebel alliance” thing we’ve been hearing so much about? It sounds subversive – we want in! Seriously though, what’s the mission of this group? Please explain it to our un-initiated readers.

Read more…

According to Twitter co-founder Biz Stone, Twitter now has 105,779,710 registered users—and is adding 300,000 new users a day. Attempting to weed through all of the fluff can be daunting! So, if you’re interested in jumping into the Twittersphere or are just looking to follow the leading journalists and thinkers in software testing today, check out our “Top 20 Software Testing Tweeps” list below (in no particular order)!

1. James Bach – @jamesmarcusbach
2. Michael Bolton – @michaelbolton
3. Testing At The Edge Of Chaos (Matt Heusser) — @mheusser
4. Tester Tested! (Pradeep Soundararajan) – @testertested
5. StickyMinds.com (Better Software Mag) — @StickyMinds
6. SearchSoftwareQuality.com (Yvette Francino) — @yvettef or @SoftwareTestTT
7. Google Testing Blog (Copeland/Whittaker) – @copelandpatrick or @googletesting
8. Testy Redhead (Lanette  Creamer) – @lanettecream
9. Test Obsessed (Elizabeth Hendrickson) — @testobsessed
10. SD Times — @sdtimes
11. Jon Bach – @jbtestpilot
12. Software Test & Performance Mag –- @STPCollab
13. Software Testing Club (Rosie Sherry) — @rosiesherry or @testingclub
14. Lisa Crispin — @lisacrispin
15. Fred Beringer — @fredberinger
16. uTest (shameless plug! ) — @uTest
17. Weekend Testing (Santhosh/Parimala/Ajay) – @weekendtesting or
18. Santhosh Tuppad — @santhoshst
19. Ajay Balamurugadas — @ajay184f
20. Parimala Shankariah — @curioustester

Update! Thanks for everyone’s recommendations. Here are a few we missed: @sbarber, @QualityFrog, @dailytestingtip, @sdelesie, @Rob_Lambert, @chris_mcmahon, @hexawise, @marlenac, @shrinik, @sbharath1012, @sellib, @TestingNews.

Please feel free to add any active Tweeps you think we may have missed in the comments! We welcome your recommendations.

Although software testing doesn’t take weekends off, our blogging team does (most of the time). So, in an effort to tide you over until Monday morning, here are a few testing related stories – each well deserving of a weekend read. Enjoy!

Tester Professionalism
From “Uncle Bob’s” post on Sapient Testing: “It seems to me that James (Bach) is attempting to define “professionalism” as it applies to testing. A professional tester does not blindly follow a test plan. A professional tester does not simply write test plans that reflect the stated requirements. Rather a professional tester takes responsibility for interpreting the requirements with intelligence. He tests, not only the system, but also (and more importantly) the assumptions of the programmers, and specifiers.

I like this view. I like it a lot. I like the fact that testers are seeking professionalism in the same way that developer are. I like the fact that testing is becoming a craft, and that people like James are passionate about that craft. There may yet be hope for our industry!”

Podcast: Matt Heusser Explains the “Rebel Alliance”
From SearchQualitySoftware.com: “No one wants to eat a bagel alone.” This is the underlying principle behind the formation of “The Rebel Alliance” a team of STAREast bound testers and developers who will attend the conference as a group. “The intention is to make everyone comfortable, introduce ourselves to the minds that we respect in software and expand our networks,” says Heusser. This kind of collaborative effort also transcends into the session Heusser will be presenting at STAREast which explains creation and service of SocialText, where Heusser is employed as a tester.”

Read more…

Last week, we found out that our humble little Software Testing Blog won the Hive Award at SXSW as the top business software blog (here’s the slideshow and the PDF report). We’re honored to make this prestigious list, along with brands we love such as HowStuffWorks, Nokia, Nike, HBO and About.com.

Part of the reason this blog has been so successful in the past year is how infrequently we talk about ourselves (ugh, boring). Well, I’m allowing myself to break that rule briefly so I can thank the people who have made our blog what it is today.

• Our in-house team (Stanton, Mike, Jenny and Peter) for their tireless efforts and talented writing about everything from mobile apps to social media to software testing to crowdsourcing trends.
• Our guest bloggers from the uTest community who have written passionately about everything from mobile testing to QA in agile environments to the evolving roles of testers.
• Our Testing The Limits guests (including James Whittaker, Matt Heusser, James Bach, Michael Bolton and Jon Bach) who have not only tolerated our wide range of questions — from the insightful to the inane — but joined in with good humor, wit, eloquence and intellect.

I’ll end this little Oscar speech before the orchestra starts playing me off stage. Suffice it to say, we love writing for you; we’ll keep scouring every corner of the world (virtual and physical) for fresh topics and angles about anything related to software; and we’ll keep reminding ourselves why we’ve had this success: we write stuff that you seem to enjoy reading. We now return you to your regularly scheduled programming.

After Twitter-stalking him, making some harassing phone calls and sending threatening letters, Jon Bach (@jbtestpilot) cheerily agreed to take part in our Testing the Limits series. Much like his brother, Jon has a remarkable understanding of software testing – both in theory and in practice. Having worked for companies like Quardev, LexisNexis, HP and Microsoft, Jon is also a blogger, author and software testing consultant. An expert, in the truest sense of the term.

In the first installment of our two-part interview, we get Jon’s thoughts on sibling rivalry; the blame spiral of software development; the emergence of “agile-fall”;  testing at a startup vs. testing in the enterprise; John Schneider as Jon Bach and more.

uTest: A few months back, we asked your buddy Andy Muns who’d win a fight between you and your brother (this was a big debate in the uTest office). He said you would win hands down. Would he be right? And since you and your brother seem to share the same testing philosophy, what would do you think the fight would be about?

JB: It’s hard to fight with someone who stayed in their room for most of our childhood.  He was either reading or doing science experiments with a microscope or the chemistry set.  It got worse when we got the TRS-80 in 1980.  In fact, that’s probably the last time we fought — over who got computer time next.  My memory may be fuzzy, but just when it came to blows, he programmed a user name and password dialog? Something clever like that. Now it’s better just to learn from him and do my best to keep up — but that’s true for all younger brothers, I think.

As for modern-day fighting, sponsor me for a testing certification and let’s see what he’d do.

uTest: Say you’re named grand poobah of the QA universe… what’s your first decree?

JB: Effective today, “Quality Assurance” is now “Quality Assistance”.

(Try it.  Watch what happens when you start using it.)

Read more…

Earlier this week, we published our three-part interview with Michael Bolton.  This was the latest installment in our monthly Testing The Limits series, in which we sit down with luminaries from the worlds of testing, development, crowdsourcing or startup life.  As part of this discussion, we asked Michael for his take on the issue of testing certifications (as we’ve done with Matt Heusser and James Bach in previous months).

In response to what she felt was “cert-bashing,” Charity Stoner of ProtoTest has written a post defending test certifications.  Since we always encourage civil discourse and open-minded debate — and since the purpose of  the Testing The Limits series is to offer up different perspectives from around the world of software — I wanted to shine a light on this post.

What do you think about test certifications?  Do they provide testers with a toolkit that complements their experience and adds real value?  Are they a marketing mechanism that limits what it means to be a professional software tester?  Or is it somewhere in the middle?  I’d love to hear your thoughts.

After we re-launched our brand in May, we decided that the uTest blog needed to be more than just uTest employees talking about uTest events, uTest awards and the uTest community (see how repetitive that gets?).

Writing witty, thought-provoking content is really hard.  And we’re pretty lazy, but fortunately we know some extremely smart & funny people.  So we invented the Testing The Limits series, in which we interview leaders from the worlds of testing, software, entrepreneurship and crowdsourcing.

We’re immensely grateful to these talented, busy people, and we have much more planned for the Testing The Limits series in 2010.  But before we flip the calendar, these posts from this year are worth another look:

June: James Whittaker – Author, Professor and Testing Evangelist at Google

July: Rosie Sherry — Founder of the UK-based Software Testing Club

August: Andrew Muns — President of Software Test & Performance

September: Jack Margo — SVP of Internet Operations of Developer Shed

October: Jon Winsor — Author, Crowdsourcing Expert, and Founder of Victors & Spoils

November: Matt Heusser — Software Testing Author, Professor and Testing Manager

December: James Bach — Software Testing Author, Teacher and Speaker

We have some great guests and ideas lined up for 2010, including software execs, QA thought leaders, and famous journalists & authors.  As always, the goal of Testing The Limits will be to inform, to entertain, and above all else, to help our readers get to know these thought leaders who are worth following and listening to.

Have a suggestion for a future Testing The Limits guest?  Drop us a note or tell us in the comments section.

Today, we finish up our “Testing The Limits” interview with Matt Heusser.  Be sure to check out part 1 of this interview.  In this installation, we’ll discuss which mobile app testing, his OS of choice, and the testing blogs and sites Matt reads.

What other blogs, sites or message boards do you read to stay on the leading edge of all things testing?
MH: I’ve got a bit of a bias there, as I write a monthly column for Software Test and Performance Magazine. You can get the PDF version for free every month. I also subscribe to Better Software Magazine. For communities, I like softwaretestingclub.com and the agile-testing discussion list. For blogs, well, there’s James Bach, Cem Kaner, and Michael Bolton. Adam Goucher has been doing a lot of writing lately, including editing the new book, Beautiful Testing, which I contributed a chapter to. And I spent a fair amount of time working on my own blog, “Testing At The Edge of Chaos.”

Recently I’ve been getting to know people by working on projects with them; my friend Chris McMahon started a Google Group on Writing About Testing which I found to be a blast. Through that group (and the Miagi-Do School) I’ve met quite a few new bloggers: Markus Gaertner, Lanette Creamer, Marlena Compton, and Ajay Balamurugadas come immediately to mind.

What OS are you running right now? What’s your browser of choice? Anti-virus? Inquiring minds would like to know.
MH: Max OS X, and I have to support all of them, so I run all browsers. Anti-Virus? Dude, I told you, I use a mac, and I try to avoid the questionable websites that host the viruses. What have you been browsing lately, Mr. Johnston?  (Ed. Note: I’ll ask the questions around here.)

Read more…

In this month’s installment of “Testing The Limits”, we sit down with Matt Heusser (@mheusser) — prolific blogger for STPCollaborative, thought leader and testing extraordinaire.  We’ll discuss the state of software testing, SpeedGeeking, the role of chaos in testing software, and the lack of fistfights at STPCon 2009

uTest:  We loved the SpeedGeeking session you led at STPCon, so we’re going to flip it on you – If you had just five minutes to teach, motivate or inspire the uTest audience about software testing, what would you say?
MH: Well, I’d start by asking the audience what they are doing today – what’s the greatest point or opportunity they feel – and asking what options they see to improve. Most of the time, I hear that testing is “too slow” or “the bottleneck” or something like that.

So I suggest taking two weeks and actually measuring how the team is spending its time. Oh, not for reporting – it is very important the team stop the time tracking after two weeks and not hand individual metrics into management for evaluation. Instead, we want to use the numbers for improvement. For example, many of the people I talk to can spend 80% of their time or more in meetings, working on documentation, working on compliance activities, doing email, and so on. That only leaves 20% of the time to test! Just pushing those numbers from 80/20 to 60/40 will double the amount of time the team spends actually doing testing.

Another thing to look at is the amount of time spent trying to reproduce defects, document defects, file bug reports, “verify” fixes, and so on. We think of these activities as testing, and they can take a substantial chunk of that 20% – but they are really accidental. That’s not a testing bottleneck – it is a development bottleneck. If test can work with development to improve the quality of the software prior to code complete, that will improve the speed of the whole system. Realizing this, and having a little bit of data to “prove” it, can help the entire system improve.

So if I had five minutes, I would say start with measuring how you track your time, and ask yourself if this is the best use of your time and what can change. Sometimes, the big boss will say “no, we absolutely need you to fill out all seven pages of documentation per test run”, and you can say “ok.”  Six months from now, when someone asks why the big project is late, you can point out that the business made an explicit decision to pay the full price of defined process. You presented options and those were not accepted.

That won’t save this project — but it might save the next.  It also turns out that actually testing tends to be much more fulfilling than documentation and compliance activities. Who could have guessed?

Lots of contrasting opinions at last month’s STP Conference. While there were no fist fights (that we heard about anyway), what did you see as the most contentious issue? And where do you fall on this issue?

Read more…

Earlier this week, several members of the uTest team took in the opening reception of Software Test & Performance’s STPCon event.  We overheard some great conversations, and even jumped into the fray a few times ourselves.  It was a lively crowd and a great venue (the Hyatt Regency) right along the Charles River.  The event continues today, but even if you’re not attending you can still follow along, check out the STPCon Twitter stream here.

This event gave us a chance to reconnect with our friends from STP Collaborative, as well as sit in on sessions from top-shelf testing thinkers like James Bach, Jon Bach, Michael Bolton and Matt Heusser.  We also got to connect with about a half dozen local uTesters who took us up on our invitation to attend the reception.

We had a few people from uTest in attendance on Thursday and they gave the presenters high marks overall.  The two that I heard the most comments on, however, were from James Bach and Michael Bolton.  Bach tackled the provocative subject of “How to Fake a Test Project.”  Bolton took on the topic of “Rapid Software Testing.”  We’ll see if we can get either of their presentations and share them here. Also of great interest was Friday morning’s SpeedGeeking session with Matt Heusser, which put the speakers on the spot, giving them only five minutes to get right to their point. Talk about fighting your natural instincts!

For anyone who attended, chime in and share your most or least favorite moments from STPCon?  What surprised you, frightened you, entertained you or just generally pissed you off? Sound off in the comments or drop us a line.

Meanwhile, the uTest paparazzi was present and ready to capture some of the scenes from the evening.

Matt Heusser and James Bach take a timeout from their testing debate for a quick photo

Matt Johnston and James Bach deep in discussion about the art & science of testing (and the difference between the two)

Jennifer Moebius and STP chief, Andy Muns take a break from the exhibits to pose for a pic