In this short interview, he explains the ideas behind what he calls “complete testing.” Good stuff. Take a look:

Want to see more interviews from STPCon? Check out the full list here.

Anyway, this is a worthwhile read for two reasons:

1. Heusser knows of which he speaks, as he’s not just a pundit. This guy has lived it, running QA organizations within large and small companies.
2. Whether they’ll admit it or not, the TCO of testing is a major concern for tech execs in all industries. So any book that tackles that issue head-on is both ambitious and timely.

For those who purchase How to Reduce the Cost of Software Testing, drop us a note and let us know what you think. We’ll publish your comments in a follow-up post.

Also, we’ll see if we can wrangle an interview with the brains behind the book (Heusser and Govind Kulkarni) in the next week or so. Want us to grill them on anything in particular? Drop us a comment and we’ll put ‘em on the spot.

Yet this concept is….well, just a concept…until it’s experienced first-hand. Matt Heusser, one of the very best testing writers out there, recently shared some details on his (first?) foray into mobile app testing with SearchSoftwareQuality. His account covers screen-size discrepancy, the expanding device matrix, GUI issues and other testing topics we all know and love.

I was particularly drawn to his “ah-ha” moment in the second paragraph (emphasis added). Take a look:

So there I was, on my iPod Touch, trying to get to a list of users whose name started with the letter “I.” It worked great on the simulator with a mouse, but with the actual iPod, my finger was too fat to click the single line of pixels.

Suddenly it hit me: This is different. Sure, all of the old GUI rules apply, but suddenly we have a new set of ways the application can fail. This tip provides a quick set of guidelines to consider, primarily for Web-based mobile applications, but much of it applies to native applications a s well.

Screen real estate

You might use a mobile device just like a regular 1024×768 pixel application, but your users probably won’t. Try to actually use the application on a number of devices — just use it. You’ll likely come away suggesting a mobile interface, perhaps an automatic re-direct on login when your application senses a mobile device. Even then, you’ll want to explore the application in a number of devices, looking for usability problems.

The supported device problems

iPods, Blackberrys, Android, and other smart phones may all be running a different browser, so complex Javascript is likely to work on one and fail on another. My recommended fix to this is similar to the one for screen real estate: Develop a simplified “mobile” interface that uses vanilla HTML.

Conversions between interfaces

If you develop a mobile interface, you’ll likely want a way for the users to switch back to the “full” interface and mobile. This should be seamless. You may even want to test the “full” interface on a mobile device, to some extent, for the sake of your power users.

Read the rest of the article >>>

This seems like an appropriate place to insert a few important reminders:

• For readers: We have an extensive library of material related to mobile app testing, which you can find here,  here and here.
• For testers: If you want to expand your skills into the world of mobile, you can search the Paid Projects section of the uTest Forums (membership is free, but required).
• For companies: If you are in need of mobile app testing, you should check out uTest Express (for startups) as well as our signature services.

In part II of our Testing the Limits interview with Lanette Creamer – aka “Testy Redhead” – we cover the need for Exploratory Testing; Matt Heusser and the “rebel alliance” of testing; how to create a popular testing blog; her stance on tester certifications and more from the wide world of QA. Catch up by reading part I, and when you’re done with this one, go check out part III.

uTest: In one of your recent blog posts, you mention Elisabeth Hendrickson’s STAREAST declaration that “Exploratory Testing Is Not Optional.” Why did this statement resonate so strongly for you? Do you think all testing managers should follow Hendrickson’s lead?

LC: As a frequent conference attendee and enthusiastic reader of testing blogs, I’ve seen many ideas about how to improve testing. I’ve been through countless industry trends, such as borrowing manufacturing ideas, extensive measuring schemes, and repeated attempts to automate all testing. The bottom line is exploratory testing works in practice. Not for a few months or years, but it works to find important bugs year after year no matter what other quality trends are happening. It works well side by side with automated checks, manufacturing ideas, and it can be used with session based test management to provide measures and metrics if needed. It is the one constant a tester can go back to and find bugs that impact the user experience. It is the meat in my testing sandwich. (My pun filled humor is the cheese.) To hear Elisabeth acknowledge the importance of exploratory testing in public shows me that agile testing is about more than just automation. Agile testing can be about a balanced approach to overall quality. It resonated with me strongly because it makes me hopeful for the future of testing on agile teams.

I think test managers should evangelize and defend what works well in practice on their teams. My experience has been that exploratory testing is generally undervalued considering how effective and practical it is.

uTest: What’s the deal with this “rebel alliance” thing we’ve been hearing so much about? It sounds subversive – we want in! Seriously though, what’s the mission of this group? Please explain it to our un-initiated readers.

LC: Our Han Solo, and Rebel Alliance leader is Matt Heusser. You may have heard of him? (Editor’s note: Oh yeah, that guy!) I’m a rebellious redhead, and was lucky enough to be invited to join, so why not? A few of us paid for our own travel and lodging so that we could swing StarEast in this economy despite diminished budgets. We pooled our resources to get some discounts, and Matt put together a dinner. He did a tremendous amount of work! Dinner plans grew. Then the force was with us, and a few Jedi mind tricks later, it turned into something incredible.

We ended up with some new lightening talks presented to a small group and having a party in a 68 person conference room that the hotel happened to assign a conference attendee as they were out of normal hotel rooms. Sitting at a table pairing on the dice game led by Michael Bolton was enlightening. Hearing Matt Heusser and Jon Bach share brand new ideas and seeing Shmuel Gerson’s session based test management test tool in action, followed by an in depth description of where combinatorics have the most impact with Justin Hunter from Hexawise was a great experience! Adam Goucher even talked about pirates! Selena recorded many of these 5 minute talks. They are now up on YouTube and you can check them out, including my talk about Cat Collaboration.

The Rebel Alliance isn’t just something you join, it is something you create at your conference. The founding principle is “no one wants to eat a bagel alone”. In practice it is how we joined together and saved hundreds of dollars so that testers without corporate backing were able to see some of the best speakers in the industry. Don’t wait for your company to send you. Send yourself! That’s what I consider the real founding principle of the rebel alliance, but hey, it’s Matt’s invention, so he gets to say it is about having company while eating bagels.

uTest: In our interview with James Bach a few months back, he called you a “great unknown” and “someone to watch.” Blogging seems to have really opened up a few doors for you, but can it do the same for other testers? What’s the secret to getting “discovered” as a testing blogger?
LC: The idea that you can “get discovered” or that I know some secret made me giggle. I’ve been blogging on software testing regularly since 2007. I go to every conference I can possibly go to. My secret to getting discovered as a blogger is to sincerely care about software testing and do good work in public. Interact. Follow up. Be on twitter. Read the books that testers you admire suggest. Write a technical paper and submit entries at some conferences. Are you willing to test in public? Are you proud to talk about your ideas to anyone? I used to send links to my own team at Adobe if I posted a blog, and exactly that many people would look at the post. Blog even if you get 30 visitors a year for a few years. Blog like no one is watching, because at first no one will be, and later because that is what the blogging format is for. You have to care more about software testing than you care about what happens to you personally. That is the one thing the testers I admire have in common.

In up and coming testers, my focus has been mostly on encouraging other female testers who I admire to begin writing and presenting. Some of the women to watch in testing are Marlena Compton, who gave her first conference presentation on visualizing test data last year, and Liz Marley, who presented on Testing Touch Devices at Adobe a few months back. There are some really intelligent Agile testers you should check out if you get the chance. Lisa Crispin, Janet Gregory, Dawn Cannan, and Selena Delesie are teaching me new things about agile testing.

A few guys to watch for are Eric Jacobson I met at StarEast a few weeks back. He’s really articulate, generous, and has a clear point of view on testing. A programmer in testing who interests me is Shmuel Gershon from Intel. His enthusiasm and good nature are infectious!

uTest: What’s your stance on tester certifications? Pro-cert, anti-cert or meh?
LC: Meh. I’d like it if as a community we could agree to a loose guideline that unless you have something groundbreaking and new to say on the topic no one else blog about this. It’s been done (to death, or at least to me wishing for death).

I don’t enjoy conversations about certification, which is why my only blog post about it is a joke. I’d prefer to work for a company that is focused on finding testing talent, not relying on someone else to determine what that is.

Editor’s note: Read part III of the interview now.

1. James Bach – @jamesmarcusbach
2. Michael Bolton – @michaelbolton
3. Testing At The Edge Of Chaos (Matt Heusser) — @mheusser
4. Tester Tested! (Pradeep Soundararajan) – @testertested
5. StickyMinds.com (Better Software Mag) — @StickyMinds
6. SearchSoftwareQuality.com (Yvette Francino) — @yvettef or @SoftwareTestTT
7. Google Testing Blog (Copeland/Whittaker) – @copelandpatrick or @googletesting
8. Testy Redhead (Lanette  Creamer) – @lanettecream
9. Test Obsessed (Elizabeth Hendrickson) — @testobsessed
10. SD Times — @sdtimes
11. Jon Bach – @jbtestpilot
12. Software Test & Performance Mag –- @STPCollab
13. Software Testing Club (Rosie Sherry) — @rosiesherry or @testingclub
14. Lisa Crispin — @lisacrispin
15. Fred Beringer — @fredberinger
16. uTest (shameless plug! ) — @uTest
17. Weekend Testing (Santhosh/Parimala/Ajay) – @weekendtesting or
18. Santhosh Tuppad — @santhoshst
19. Ajay Balamurugadas — @ajay184f
20. Parimala Shankariah — @curioustester

Update! Thanks for everyone’s recommendations. Here are a few we missed: @sbarber, @QualityFrog, @dailytestingtip, @sdelesie, @Rob_Lambert, @chris_mcmahon, @hexawise, @marlenac, @shrinik, @sbharath1012, @sellib, @TestingNews.

Please feel free to add any active Tweeps you think we may have missed in the comments! We welcome your recommendations.

Tester Professionalism
From “Uncle Bob’s” post on Sapient Testing: “It seems to me that James (Bach) is attempting to define “professionalism” as it applies to testing. A professional tester does not blindly follow a test plan. A professional tester does not simply write test plans that reflect the stated requirements. Rather a professional tester takes responsibility for interpreting the requirements with intelligence. He tests, not only the system, but also (and more importantly) the assumptions of the programmers, and specifiers.

I like this view. I like it a lot. I like the fact that testers are seeking professionalism in the same way that developer are. I like the fact that testing is becoming a craft, and that people like James are passionate about that craft. There may yet be hope for our industry!”

Podcast: Matt Heusser Explains the “Rebel Alliance”
From SearchQualitySoftware.com: “No one wants to eat a bagel alone.” This is the underlying principle behind the formation of “The Rebel Alliance” a team of STAREast bound testers and developers who will attend the conference as a group. “The intention is to make everyone comfortable, introduce ourselves to the minds that we respect in software and expand our networks,” says Heusser. This kind of collaborative effort also transcends into the session Heusser will be presenting at STAREast which explains creation and service of SocialText, where Heusser is employed as a tester.”

Tester Shortages Being Reported
From James Hutchinson’s latest piece in ComputerWorld: “As companies begin to revise and implement IT projects put on hold during the economic downturn, software testing companies and consultants have seen an increase in demand for their niche services. However, some report they are struggling to keep up with increased demand, and have begun looking to overseas markets to to bring in more skilled testers.”

iPad, iProblems
From Rafe Needleman’s column: “Everyone knows the first-gen iPad is lacking a camera and multitasking and that many of its apps are overpriced. Annoying. But those of us who bought iPads knew this going in, and we’ve found ways to work around or to justify to ourselves these omissions in the product. What we didn’t know about on iPad Day, April 3, was all the little things that would drive us up a tree. These are the annoyances and roadblocks that makes this appliance, otherwise engaging and attractive, feel like it was rushed out the door. Or done on the cheap.”

Have a great weekend everyone!

Part of the reason this blog has been so successful in the past year is how infrequently we talk about ourselves (ugh, boring). Well, I’m allowing myself to break that rule briefly so I can thank the people who have made our blog what it is today.

• Our in-house team (Stanton, Mike, Jenny and Peter) for their tireless efforts and talented writing about everything from mobile apps to social media to software testing to crowdsourcing trends.
• Our guest bloggers from the uTest community who have written passionately about everything from mobile testing to QA in agile environments to the evolving roles of testers.
• Our Testing The Limits guests (including James Whittaker, Matt Heusser, James Bach, Michael Bolton and Jon Bach) who have not only tolerated our wide range of questions — from the insightful to the inane — but joined in with good humor, wit, eloquence and intellect.

I’ll end this little Oscar speech before the orchestra starts playing me off stage. Suffice it to say, we love writing for you; we’ll keep scouring every corner of the world (virtual and physical) for fresh topics and angles about anything related to software; and we’ll keep reminding ourselves why we’ve had this success: we write stuff that you seem to enjoy reading. We now return you to your regularly scheduled programming.

After Twitter-stalking him, making some harassing phone calls and sending threatening letters, Jon Bach (@jbtestpilot) cheerily agreed to take part in our Testing the Limits series. Much like his brother, Jon has a remarkable understanding of software testing – both in theory and in practice. Having worked for companies like Quardev, LexisNexis, HP and Microsoft, Jon is also a blogger, author and software testing consultant. An expert, in the truest sense of the term.

In the first installment of our two-part interview, we get Jon’s thoughts on sibling rivalry; the blame spiral of software development; the emergence of “agile-fall”;  testing at a startup vs. testing in the enterprise; John Schneider as Jon Bach and more.

uTest: A few months back, we asked your buddy Andy Muns who’d win a fight between you and your brother (this was a big debate in the uTest office). He said you would win hands down. Would he be right? And since you and your brother seem to share the same testing philosophy, what would do you think the fight would be about?

JB: It’s hard to fight with someone who stayed in their room for most of our childhood.  He was either reading or doing science experiments with a microscope or the chemistry set.  It got worse when we got the TRS-80 in 1980.  In fact, that’s probably the last time we fought — over who got computer time next.  My memory may be fuzzy, but just when it came to blows, he programmed a user name and password dialog? Something clever like that. Now it’s better just to learn from him and do my best to keep up — but that’s true for all younger brothers, I think.

As for modern-day fighting, sponsor me for a testing certification and let’s see what he’d do.

uTest: Say you’re named grand poobah of the QA universe… what’s your first decree?

JB: Effective today, “Quality Assurance” is now “Quality Assistance”.

(Try it.  Watch what happens when you start using it.)

uTest: When there are delays in development process, why does testing always seem to take the blame? Is their role just misunderstood? Or is it really the testing team’s fault?  How can companies avoid this seemingly never-ending spiral?

JB: Often, we’re saved until last.  It’s like the novel is written, the movie is produced, but the reviews aren’t in yet, leaving the creators in a heightened state of worry and concern. It’s a lot of stress to be in that position. As testers shine the light on the product, looking for risks and vulnerabilities, the light is really shining on *us*.  We’re being tested just as much as that software, so we tend to be sensitive and aware of being in a critical position as others wait for our findings.

Not being an Agile guy, I’m finding that the values and principles of Scrum, Agile, Lean, and XP may be compelling enough to try.  I’m getting more into that domain and I see merit in the values and principles it is trying to instill.  One of my favorite colleagues (Elisabeth Hendrickson) is coaching me in a gentle way and her stories are enlightening because she lives this stuff. Like me, she used to focus on exploratory testing, but catching up with her recently, she seems to be the whole package — developer, tester, manager, coach — and seems to be immune from that never-ending blame spiral you asked about.  That’s compelling to me.  Some may call her one of those ardent “Agilista” types, but to me, she hasn’t lost her testing spirit or soul.  Though she may agree there’s no silver bullet, she has some great experiences that convince me that with those approaches, the blame spiral is being made irrelevant because of the way testers are more involved, earlier.

uTest: In Half-Baked Ideas for Rapid Test Management (PDF) you used a term called “Agile-fall” (a combination of Agile and waterfall). This seems to be the methodology that most companies follow, yet they always call themselves agile. Is there any shame in being Agile-Fall? And did you coin this term? We couldn’t find it anywhere else.

JB: “Agile-fall” was something I heard at LexisNexis from an awesome PM named Lance Thomas, but in a Google talk in 2005, Elisabeth Hendrickson called it “Scrumfall”, so search on that term and you’ll see that it refers to having the principles associated with Agile (daily stand-ups, sprints, burndown charts, etc.), done in a waterfall-y series of development steps.  Example: Sprint 1: Gather requirements, Sprint 2: Design your tests, Sprint 3, Run those tests, Sprint 4, Fix bugs, Sprint 5 regress those bugs.  There’s no shame in that if that’s what works, and when you’re going through a transition from Waterfall to Agile, that may be the best thing as opposed to a sudden lever-pull one day where you show up and your desk is next to someone else with no walls and there’s a stack of sticky notes and markers on your chair with an email to report to your first standup in 30-minutes.

uTest: It looks like software bugs are partly to blame for the recent Toyota recall debacle. Is this the worst nightmare for testing managers? What else keeps them up at night?

JB: I almost got a Toyota last month, and wish I had, just so I could find a cool defect.  But my nightmares in testing are: “Why didn’t you catch that bug?” Though I know several answers to that, and my favorite colleague Michael Bolton has a rich list of answers to that question, they don’t often overcome the strong emotional attachment that stakeholders have.  And I can’t blame them, really.  The more we call ourselves “Quality Assurance” (like we can guarantee quality), the more they will lean on us to assure them.  I can’t.  But I can certainly *assist* with the notion of  whether something has value to its intended customer.

What keeps me awake is how to know if I’m bringing the right value to the project.  There are a million things I could do, but which ones have the most value right now?  I’m worried I would stumble into an idea too late, or not at all.  That’s why I like heuristics and mnemonics and checklists to kick my mind into gear when I feel stuck or worried.

uTest: What’s the biggest difference between working for a large, mature enterprise and a small young startup (from a testing point of view, that is)?

JB: I’ve worked at both.  At Quardev, which is neither startup or enterprise, we’ve got a mature-but-startup mentality.  Since the word “Quardev” comes from an amalgam of QUality Assistance, Research, and DEVelopment, the balance of those gives us enough diverse opportunities to make it feel like a big company.

From a testing point-of view, test ideas are king no matter what size your company. Some may say the toolset is king, or dev skills are king, but when I’ve worked for start-ups, you don’t need as many signatures on things, and that helps creativity. You might fail faster with those test ideas because you have to do it cheaply, and that’s good.  It’s like projects I’ve been on that use notions of Agile compared to those that don’t.  I see those kinds of projects revealing problems faster because of the way the culture organizes to deliver working software.

At a small startup, it’s often expected that you’ll be brilliant.  That pressure can make work not-so-fun.  Maybe that’s why some of my best ideas came from working in enterprise environments (Microsoft, HP, LexisNexis). In those places, there was a perception that every test idea, method, tactic, and strategy had been tried before.  That carried an unspoken dare (in my view) that begged to be challenged, which few people took on.  For me, brilliance and innovation come when someone counts me out or doesn’t expect it.  I like that.  I like exceeding expectations no matter how big or small the crew.

What sums it up is a TED talk by Ken Robinson, who said: “If you’re not prepared to be wrong, you’ll never come up with anything original.”

uTest: Who plays Jon Bach in the made-for-TV-movie about your career in testing? And what’s the title?

JB: John Schneider, only because it has to be called “The Duke of Hazards.” It also fits because I told my friends in school that I was related to Cousin Daisy (played by Catherine Bach). Not true, but it helped keep the bullies away.

Editor’s note: Here’s part II of the interview.

In response to what she felt was “cert-bashing,” Charity Stoner of ProtoTest has written a post defending test certifications.  Since we always encourage civil discourse and open-minded debate — and since the purpose of  the Testing The Limits series is to offer up different perspectives from around the world of software — I wanted to shine a light on this post.

What do you think about test certifications?  Do they provide testers with a toolkit that complements their experience and adds real value?  Are they a marketing mechanism that limits what it means to be a professional software tester?  Or is it somewhere in the middle?  I’d love to hear your thoughts.

Writing witty, thought-provoking content is really hard.  And we’re pretty lazy, but fortunately we know some extremely smart & funny people.  So we invented the Testing The Limits series, in which we interview leaders from the worlds of testing, software, entrepreneurship and crowdsourcing.

We’re immensely grateful to these talented, busy people, and we have much more planned for the Testing The Limits series in 2010.  But before we flip the calendar, these posts from this year are worth another look:

June: James Whittaker – Author, Professor and Testing Evangelist at Google

July: Rosie Sherry — Founder of the UK-based Software Testing Club

August: Andrew Muns — President of Software Test & Performance

September: Jack Margo — SVP of Internet Operations of Developer Shed

October: Jon Winsor — Author, Crowdsourcing Expert, and Founder of Victors & Spoils

November: Matt Heusser — Software Testing Author, Professor and Testing Manager

December: James Bach — Software Testing Author, Teacher and Speaker

We have some great guests and ideas lined up for 2010, including software execs, QA thought leaders, and famous journalists & authors.  As always, the goal of Testing The Limits will be to inform, to entertain, and above all else, to help our readers get to know these thought leaders who are worth following and listening to.

Have a suggestion for a future Testing The Limits guest?  Drop us a note or tell us in the comments section.