Software Testing Blog

Well we certainly had a wicked good time at QAI QUEST (Quality Engineered Software & Testing Conference), this year on our very own soil, the “Hub of the Universe” — Boston! From the uTest tutorial on “Everything You Need To Know About Mobile App Testing,” to the amazing reception at the Seaport Hotel where we played human scrabble, to the eclectic expo, to Doron’s presentation on “The Top 10 Disruptive Trends Altering Testing,” to signing the hilarious Declaration of Independence from Software Defects — Beantown provided the ideal setting for this testing pahhty! See more pics above.

And of course we can’t let you go without sharing some insights from QUEST on today’s hottest topics and emerging trends in software QA and testing. As a bonus, the QUEST Magazine was distributed to all registered attendees and provided everyone with some great, original articles on testing. I’ve included it below. Check out eBay’s Jon Bach’s article on Transition and Exploration, uTest’s Doron Reuveni’s article on The Great Quality Challenge for Mobile Apps, HP’s Rafal Los’ article on Security and Requirements, and more! Complete magazine [pdf] after the bump.

Read more…

In part II of our Testing the Limits interview with Jon Bach, we get his thoughts on responding to change in the testing world; what his brother James Bach has been up to; his criteria for hiring testers at eBay; mobile challenges; searching for defective pocket change and more. If you missed the first session, you can read it here.

uTest: It looks like eBay wasn’t able to keep you off the testing speaker circuit (woohoo!). In fact, you were at STPcon earlier this month – care to give our readers who couldn’t make it a summary of what you covered?

JB: Two things: A workshop with Dan Downing of Mentora, who approached me at the last WOPR (held at eBay in November) and had a cool idea to bring a little slice of WOPR to STP.  It’s for anyone who needs to build a game plan for performance testing.  He called it “Arming Yourself for Performance Testing: War Stories from the Trenches” — http://www.stpcon.com/Item/1032/.

I also spoke about an idea that I’ve been experimenting with after James came back from a business trip and talked about how to respond to project change and chaos: http://www.stpcon.com/Session/13/My-Crazy-Plan-for-Responding-to-Change

uTest: Speaking of James, he has been doing some interesting things the past year as well. What’s the latest testing topic of conversation among the Bach brothers? And did he have any words of advice for you in starting your new job?

JB: He came to eBay and spent a week with me.  I sat him in the cube next to mine and he did some testing from outside the firewall on the guest wireless. I gave him a charter and he executed it beautifully.  The secret about James is, he’s really friendly and service-minded if you’ve managed to win his respect.

We talked CAST 2011 (I’m conference president, he’s my program chair); we talked about new tester games; we shot a new CAST promo video; we talked about Egyptian democracy and systems thinking (how it affects the price of gas).  But just when we were in the thick of testing eBay site page compatibility with IE 9.0, the Japan quake hit and we took time to watch the footage with the rest of the world.  Then we did impromptu research and found out more on nuclear plant  meltdowns, which led to being curious about microseiverts, which led to an article about Byzantine failures.

About eBay, he gave me no advice per se, just ideas for tactics.  He offered some free consulting, which he gave, then said, “I’m proud of you, man. Kick ass.”

uTest: Part of your new role at eBay will be to hire and recruit a top-flight team of testers (in addition to the ones already there). What sort of traits/skills/attributes will you be looking for in particular?

JB: The ability to come up with ideas – either old or new – and execute them in a way that helps us improve notions of Search.  For years, I used the triangle program in test auditions.  Now I use something more simple.  I draw a long horizontal rectangle on the whiteboard with a little “Submit” button below that.  I say “this is a text input field for Search, just like the one you see on the eBay site. Help me create a test plan for it.”  I’m hoping that instead of an interview, it comes across more like an invitation to a real collaboration.

Read more…

Sometimes, we interview a testing expert with so much wisdom, so much experience and so much to say, that one appearance simply won’t do. Jon Bach is one of those experts – and he’s back for his second Testing the Limits interview (Bach to the Future…anyone?). Jon recently joined eBay as the new QA Director for their Buyer Experience department. Prior to his new role, he had stints at Quardev Laboratories, Microsoft, Rational, LexisNexis, and Hewlett-Packard. Impressive track record, eh?

In part I of the interview, Jon discusses his new role at eBay; knowledge transfer in testing; war stories in software testing; how to test with no requirements and more. When you’re done here, go read Part II.

*****

uTest: First off, congrats on your new job as the QA Director for eBay. I know you’re only a few weeks in, but what are you most looking forward to in this new role? And how is the transition going so far?

JB: I work for an aptly-named eBay department: “Buyer Experience: Search and Discovery”.  My team handles anything that has to do with what we call “Search Front End” – the UI elements rendered and functions that support searches for merchandise on eBay. eBay does more searches per day than Google, and I’m looking forward to finding search issues that affect millions of people every day.

As for “transition”, I have my laptop, badge, and a name tag for my office; bought myself an eBay hat, but the most important “transition” is one of mindset.  I used to think eBay was kind of like a pawn shop or a swap meet, but it’s got millions of new items from major manufacturers.  It’s a platform for worldwide ecommerce, whether it’s items sold by individuals or major retail outlets.

The most important transition for me is changing my own ecommerce habits.  Instead of going to Amazon to buy books, for example, it’s more useful for me as a tester and as an eBay employee to think “can I get it from eBay first?” Recently, I needed to order Valentine’s Day flowers for my wife.  I started going to proflowers.com when I thought, “can I get them through eBay?” Turns out that eBay options for flowers are not displayed very prominently on the site.  I finally found text ads for florists on the bottom of one of eBay’s search results pages, but I had to dig to find them.  This caused me to email my PMs – “why don’t we have a more prominent upsell for flowers, especially on the home page for V-day?” That mindset shift inspired me to actually test the main mission of my department in terms of my “buying experience.”

uTest: It seems that knowledge transfer must be especially difficult in the testing space. As someone going through the “on-ramping” process at a big company, with lots of moving parts, what advice you have for other testers and managers in the same situation?

Read more…

Okay, call this a bait and switch if you will, but the bottom line is you cannot test quality into an application. So if you can’t test quality into an app, do you then build it into an app? Or perhaps the more pertinent question is, ‘who contributes more to app quality – software developers or software testers?’ Playing with dynamite here, I know…

Let’s begin with a simple fact – developers are the ones who “create” software defects in the first place. To be fair, they don’t knowingly create buggy software, but that’s the widely accepted norm – we’re human after all. However, when bugs are discovered after the product launches, testers are typically singled out and blamed. Why?

Part of the reason is due to the misnomer that QA should stand for “quality assurance.” Do QA professionals truly assure the quality of a product, or do they assist in delivering high quality products (as Jon Bach has suggested)? So if you’re a tester by trade, I sympathize with you. On the one hand, buggy software leads to job security. On the other hand, you are constantly on the hot seat and looking over your shoulder, wondering when and where the next bug will surface. But instead of despairing over these details, testers should rise to the challenge.

Here are a few examples of how testers can lead the quality initiative:

Read more…

The first rule of Fight Club is: you do not talked about Fight Club. Lucky for us, that rule does not apply to the Context-Driven School of Software Testing.

In case you hadn’t noticed, the context-driven school has amassed a global following in just a few short years years, despite some initial confusion on the part of newbies…What is a context-driven tester? What is the basic premise? How is it different from the other prominent “schools” of testing? And what does one have to do to become a member?

James Bach – the founding father of CDT – posted a great overview of the principles this past weekend in an article titled “The Dual Nature Of Context-Driven Testing.” He offers some key distinctions on what the term means, what it doesn’t mean, and how you can grow as a tester by learning more about its principles. Here are a few important excerpts (emphasis mine), beginning with an abridged definition:

The Context-Driven School of software testing is a way of thinking about testing, AND a small but world-wide community of like-minded testers. There are other, larger, schools of testing thought. But CDT represents my paradigm of testing. By paradigm, I mean an organizing worldview, an ontology, a set of fundamental beliefs.

CDT is not a style of testing. It’s not a toolbox of methods. It’s more fundamental than that. You could think of  CDT partly as an ethical position about testing. All methods or styles are available to Context-Driven people, but our selection of methods and reactions to testing situations are conditioned by our ethical position. This position is defined here.

Reading further, it occurred to me that the context-driven school is well-represented on the uTest Blog. To illustrate this alliance, I’ve included links to the names of those “Jedi Knights” who have made contributions on this site. Here’s the excerpt:

Read more…

In part II of our interview with Elisabeth Hendrickson (aka @TestObsessed), we discuss the influence of her testing colleagues; the difference between a great SCRUM master and terrible SCRUM master; the lack of women in software testing; her secret talents and more. If you missed our previous segment, you can read part I here.

uTest: As a frequent guest on the speaker circuit, you’ve crossed paths with many smart people in testing. How has the “collective brain” helped you throughout your career? Care to mention anyone in specific?

EH: I’m immensely grateful to our community for their openness and generosity in sharing ideas and knowledge. I cherish the conversations with folks whether we’re meeting up in person at events or virtually through discussion lists, Twitter, etc.

And yes, there have been several people to whom I am particularly grateful for inspiring, encouraging, and challenging me. Cem Kaner taught me a huge amount and encouraged me at every turn. James Bach challenged me and in doing so forced me to hold myself to a higher standard. Karl Wiegers was astoundingly generous in sharing his lessons learned as a consultant when I was getting started. Tim Lister inspired me when he asked me “Why are your pants on fire?” at a key moment in my career. Jerry Weinberg taught me more than I can express. I could go on. Just thinking about people with whom I’ve enjoyed discussing and collaborating brings more faces to mind: Dale Emery, Lisa Crispin, Janet Gregory, Jennitta Andrea, Jon Bach, Bret Pettichord, Fiona Charles, Karen Johnson, Harry Robinson, Pradeep Soundarajan, and others. I feel blessed to be a part of such a fabulous community of people.

uTest: What characteristics or traits make for a truly great SCRUM master? What about a truly terrible SCRUM master? It seems like the type of role that is not suited for everyone.

EH: Good Scrum Masters I’ve seen give the team lots of room to be self-directing. Yes, Scrum Masters guide their teams in adopting agile practices. They advise. They remove impediments. But the most important thing they do is to create space for the group to learn, grow, and ultimately succeed as a self-organized team. The best Scrum Masters have such a light touch that it’s difficult to see what they actually do. It’s like being a good facilitator. Good facilitators get out of the way and let the conversation happen, only intervening when it’s absolutely necessary. Similarly, good Scrum Masters get out of the way of progress, trusting the team to find a path forward.

On rare occasions I’ll see Scrum Masters who misunderstand their role and want to make the process all about them. They see their role as managing the team. They take a directive stance, assigning work to team members. They conduct the Daily Scrum as a daily status ceremony. People on teams run by such a Scrum Master often report that agile feels like a particularly bad form of micromanagement. That’s a sign that the Scrum Master is completely missing the power of self-organizing teams. In doing so, they systematically disempower the team members and prevent the team from reaching its full potential. The end result is generally a sad parody of real Scrum with none of the benefits.

uTest: We can’t help but notice the major discrepancy in the male-to-female ratio when it comes to testing (and dev too, for that matter). What are the major challenges and opportunities of being a woman in this space? What could change the direction of this trend?

Read more…

We write frequently on the subject of what it takes to become a top tester – in both the uTest community and the industry as a whole. We ask the testing giants their thoughts on the matter (see quotes below) and publish guest posts and Crash Courses in an effort to help you become a better software tester.  Please hold your applause.

But what if software testing isn’t for you? What if after all the education, training and job-searching, you discovered that you really had no knack for the craft? Wouldn’t it have been nice to know that a little sooner? Lucky for you, I’ve designed this picture quiz as a humorous supplement to the Jung Career Indicator Test. Here’s how it works: If you don’t see anything wrong with these photos, then software testing is definitely NOT for you. Far from scientific, but hey, it’s a start.

“Almost all of the best people I know in testing have significant experience in other fields. It’s common for people to move from testing to programming or writing or marketing and then back, bringing what they’ve learned with them, to test with a richer perspective and with a much more productive vision of where testing can fit within development/marketing/support cycles.” – Cem Kaner

Read more…

In part II of our Testing the Limits interview with Ben Simo, we’ll discuss whether you should trust automated testing tools; the proliferation of testers on Twitter; the true meaning of “QA”; how testing evolves differently in each company; the long lost Bach brothers and much more. You can catch up on the conversation by reading part I. We’ll wrap things up tomorrow with part III.

********

uTest: Jon Bach mentioned that changing the meaning of “QA” to Quality Assistance would help outsiders (engineers, executives, et al) better understand the role of this discipline.  Agree or disagree?

Simo: I believe I first heard  “Quality Assistance”  from Cem Kaner.  I agree with Jon. When testers bear the title Quality Assurance, it often implies that they actually assure the quality of other people’s work. Testers are in a position to help assist quality; not assure it. Let’s not assist the setting of unrealistic expectations with inappropriate titles.

uTest: While we’re on the subject, are you anyway related to James and Jon Bach? The resemblance is uncanny.
Simo: I don’t think so. I’m available for adoption if the Bach family is interested.

uTest: You’ve said that you frequently use automated tools, but that you don’t trust them entirely (back to that whole defensive pessimist thing again). What advice do you have for testers and managers wanting to strike a healthy balance? And what’s currently in your arsenal of automated tools?

Simo: My mistrust in tools is based on the fact that tools can’t think for me. Automated checking can only process whatever decision rules someone thought to program when the checks were created. Automation will consistently do what it is programmed to do and consistently not do what it is not explicitly programmed to do. I find test automation to be useful. In fact, there are some things I’d not want to even try to do manually. I do, however, distrust the green bar. When automated checking passes, I ask myself what the automation does not tell me. I also try to keep aware that people who don’t understand what the automation does are likely to assume that it does more than it does.

Tools are much more than test automation. Tools are essential for testing. I don’t want to test without tools. I have some old programming books that promote testing in which a programmer manually executes code, step-by-step, with pencil and paper in order verify that the code works as expected. This is manual testing. This is a testing practice that came from a time when computer time was rare and cost more than people. We’d now laugh at someone proposing testing in this manner.

Read more…

Imagine a world where software testers are courted and wooed like LeBron James; where online job boards are littered with “Testers Wanted” posts and where everyone can finally spell “QA” correctly. In other words, imagine a world with a shortage of software testers…

“Nonsense!” you say. “There’s plenty of software testers to go around.” Not for long, says SiliconIndia, who posits that a shortage of skilled software testers is only a matter of time. Citing various facts, figures and estimates from a recent Gartner study, the author examines the reasons behind this coming tester drought.

Pradeep Chennavajhula explains:

This shortage is now a major concern for the IT service organizations, considering that the academia is not geared up to support the program, and many of the training organizations are not geared up to meet the demand of the industry. In this scenario, the question still remains as to how is the industry planning to tackle the shortage of software testers?

Good question. Of course, we’ve dealt with these issues many times before on The uTest Blog. Here are a few posts with some answers:

Read more…

According to Twitter co-founder Biz Stone, Twitter now has 105,779,710 registered users—and is adding 300,000 new users a day. Attempting to weed through all of the fluff can be daunting! So, if you’re interested in jumping into the Twittersphere or are just looking to follow the leading journalists and thinkers in software testing today, check out our “Top 20 Software Testing Tweeps” list below (in no particular order)!

1. James Bach – @jamesmarcusbach
2. Michael Bolton – @michaelbolton
3. Testing At The Edge Of Chaos (Matt Heusser) — @mheusser
4. Tester Tested! (Pradeep Soundararajan) – @testertested
5. StickyMinds.com (Better Software Mag) — @StickyMinds
6. SearchSoftwareQuality.com (Yvette Francino) — @yvettef or @SoftwareTestTT
7. Google Testing Blog (Copeland/Whittaker) – @copelandpatrick or @googletesting
8. Testy Redhead (Lanette  Creamer) – @lanettecream
9. Test Obsessed (Elizabeth Hendrickson) — @testobsessed
10. SD Times — @sdtimes
11. Jon Bach – @jbtestpilot
12. Software Test & Performance Mag –- @STPCollab
13. Software Testing Club (Rosie Sherry) — @rosiesherry or @testingclub
14. Lisa Crispin — @lisacrispin
15. Fred Beringer — @fredberinger
16. uTest (shameless plug! ) — @uTest
17. Weekend Testing (Santhosh/Parimala/Ajay) – @weekendtesting or
18. Santhosh Tuppad — @santhoshst
19. Ajay Balamurugadas — @ajay184f
20. Parimala Shankariah — @curioustester

Update! Thanks for everyone’s recommendations. Here are a few we missed: @sbarber, @QualityFrog, @dailytestingtip, @sdelesie, @Rob_Lambert, @chris_mcmahon, @hexawise, @marlenac, @shrinik, @sbharath1012, @sellib, @TestingNews.

Please feel free to add any active Tweeps you think we may have missed in the comments! We welcome your recommendations.