Software Testing Blog

Are you a software tester or QA engineer/manager? If so, a typical workweek likely involves hours and hours of scrubbing through long lists of bug reports. Tasks include identifying which ones have been fixed, which need immediate attention, which need to be reproduced, and which “need more information” from the submitter. Sound familiar?

As someone who analyzes these reports, the last thing you need is a generic and ambiguous bug report title: “crash,” “fail,” and “poor design” just to name a few. This blog post highlights a few common reasons and practical resolutions for generic bug report titles – feel free to forward this to infamous perpetrators inside and outside your organization.

So why do people report such horrible bug report titles? Here are some of the popular reasons:

• Lack of thorough knowledge about the application under test – self-explanatory and often used.
• Lack of incentive. Being thoughtful with bug report titles may not benefit the submitter at all (“what’s in it for me?”).
• Laziness. After all, it’s much easier to write a generic title than to be descriptive yet concise, which is a skill that takes practice.

With these problems identified, here are some practical recommendations:

Read more…

Want to become a better tester? These ten steps will get you there – guaranteed!

1. Test.
2. Test more.
3. Test even more.
4. Test even more than that.
5. Test when you don’t want to.
6. Test when you do.
7. Test when there are lots of projects.
8. Test when there are not.
9. Test every day.
10. Keep testing.

This list was adapted from Brian Clark’s 10 Steps to Becoming a Better Writer (link). So what advice do YOU have for becoming a better tester? Let us know in the comment section.

Sometimes there are features that just can’t be tested until the product is launched. For example, today I had a conversation with one of the uTest product managers about some upcoming features, and she lamented that there was this one thing she just couldn’t get tested. It’s an internal feature that changes the behavior for emailing our project managers, but testing it is nearly impossible because email is disabled on our staging server. Enabling emails would mean sending garbage messages to 35,000 testers, so this one feature won’t be tested until the last minute when it’s ready for launch. It’s not ideal, but it is understandable.

Another example: Robert Scoble recently interviewed Trey Ratcliff (one of my favorite photographers) about his new photo editing app for the iPad. Trey remarked that because there is no camera on the iPad 1, they had to “blindly” add the feature for taking pictures using the built-in camera – that is, without testing. That’s because when they wrote the feature, the iPad 2 (which includes a camera) didn’t yet exist. Kind of scary, but understandable. No amount of testing in the world would validate that feature until the iPad 2 went on sale.

The sad reality is that there are times when getting 100% testing coverage is impractical. Fortunately, there are some strategies we can use to mitigate any problems that may come up:

Read more…

QUEST, one of the top software testing conferences, will be held in Dallas this year (April 19-23).  And uTest is getting geared up and is thrilled to be a part of this conference.

In addition to inviting Doron to be a keynote presenter, QUEST features a week-long agenda packed with more than 100 opportunities for attendees to build new skills and prepare for the testing professions of the future.

From exploratory testing to test automation to security audits to crowdsourced testing,  QUEST will cover a wide range of testing topics that give attendees insight into the latest best practices and innovative approaches to testing today. To learn more, here’s a sneak peek at the QUEST Magazine.

Special Note: Members of the uTest community interested in registering for QUEST are eligible for

Read more…

Calling all New England QA and software testing professionals!

We will be co-hosting a free tester meetup with STP (Software Test & Performance) as part of the kickoff reception for their big event, STPCon 2009 at the Hyatt Regency Cambridge.  This meetup will be Wednesday, October 21 at 5:30pm.

Join us for a great evening of networking that will be held in the STPCon exhibits area. There, you’ll have the opportunity to connect with your peers, connect with execs from uTest and STP, discover new products and features and talk to the experts who created them.

Another great perk for attendees is that you’ll have the opportunity to discuss the latest and greatest trends with industry leaders such as James Bach and Michael Bolton.

If you’re around, it would be great to meet you in person!  To register, please visit: http://utest2009stpcon.eventbrite.com/.

Your computer just crashed taking with it that document you’ve been working on for the past hour.  Naturally, you cry out in anguish over all those terrible software bugs that conspire to crash your computer and drive you nuts. But new evidence suggests that something else could be the cause: bad memory.

Google just announced the results of a new study based on the thousands of servers they manage in their data centers, and the results are surprising.  Memory error rates are thousands of times higher than anyone previously believed.  (CNET has a good summary of the research here.)

For software testers, this represents a significant new wrinkle in identifying bugs.  Wonky behavior and computer crashes could really be a hardware problem, and these sorts of hardware issues are way more common than we previously thought.

So what does that mean for your testing?  Here are a few thoughts that could help.

Read more…

Frequent readers of the uTest blog are by now aware that we’re big fans of James Whittaker – software testing expert, author and now one of Google’s top QA guys. Over the past few weeks, James has been writing a provocative series called the “Seven Plagues of Software Testing”. You can find it on his new blog.

As you’ll notice, only six of the plagues have been published thus far. The seventh and final software testing plague was intentionally omitted, as he is accepting submissions from his readers.

Before you send him your suggestions (we included an email address at the bottom of this post), here’s a few excerpts from some of the plagues he’s discussed so far:

The Plague of Blindness: “Software testing is much like game playing while blindfolded. We can’t see bugs; we can’t see coverage; we can’t see code changes. This information, so valuable to us as testers, is hidden in useless static reports. If someone outfitted us with an actual blindfold, we might not even notice.”

Read more…

In my kitchen I have a mandoline.  No, it’s not a musical instrument (that’s a mandolin), but instead a fancy food slicer that can thinly cut or dice just about anything.  Anything, that is, including my fingers.

You see, I am extremely clumsy with my mandoline.  Every time I’ve used it, I have somehow sliced open one of my fingers.  My mandoline is literally covered in knives – there are knives for slicing, knives for dicing, and even more knives stored underneath and inside the thing for torturing vegetables in additional ruthless and unimaginable ways.  The whole thing is one big vegetable and finger slicing machine.

The crazy thing about my mandoline is that despite the fact that it cuts me every time I even look at it, it’s not actually broken.  In fact, it’s in great shape.  All of its knives are sharp, its frame is sturdy, and it slices and dices just like one would expect.  I’m sure there is some kind of spec somewhere that says “verify mandoline can smoothly cut onions exactly .125 inches (3.17 mm) thick.”  I’m sure mine passes with flying colors.  What the spec probably doesn’t say is “verify mandoline won’t cut fingers.”

Specs can’t capture these sorts of negative tests very well.  You can write them, but a spec like that is a lot like a spec that says “make sure your software doesn’t crash.”  It’s very nice to include, but also kind of worthless.  This is where good testers can really show their worth.  A tester can think beyond the spec and relay feedback that can be just as valuable as any bug or error.

What are some of the evil mandolines in your projects?  If you relayed them to your developers or product managers, do you think you could make your software better?

I recently returned from a short and relaxing vacation to Iceland.  It’s a beautiful country and well worth visiting for anyone looking to travel to a place that’s both modern and rugged at the same time.  The scenery is extraordinary and the people are all very friendly.

Now that I’m back, I thought I would write six testing lessons that I learned while in Iceland:

1.) Read the Guide!

Even though Iceland is a small country, it’s absolutely critical for anyone traveling there to first read the travel guides.  Many of Iceland’s roads are unpaved, poorly maintained, or worse.  Without reading the guides, you could end up fording a deep river or sliding through the muck in a Toyota Corolla.  Vehicle loss due to “floating downstream” is not covered by many rental agreements.

Software testing is exactly the same.  Do your homework on a release by reading the release details, testing guides, known bugs, etc.  That way you won’t end up with a flat tire in the middle of nowhere on a dirt road last used by Viking explorers.

Read more…

In the second part of his blog post “Security Testing Tips From a Bug Battle Winner”, uTester Bernard Lelchuk takes a closer look at some of the more effective tools to use when performing security testing.

There are quite a few attacking testing tools which can make security testing easier and more productive for both novice and veteran testing engineers alike. I will not list all of them here,  but rather cover the most essential, common and interesting FREE tools. So here they are, in no particular order:

Wireshark
A comprehensive yet easy-to-use protocol analyzer (sniffer) which will allow you to view, filter and analyze all network transmissions. (http://www.wireshark.org/)

Paros Proxy
Acts as a proxy which allows the tester to intercept and modify all HTTP/S data between server and client, including cookies and form fields. (http://www.parosproxy.org/index.shtml)

Burp Suite (Man-In-The-Middle)
Integrated platform for attacking web applications which contains several interfaces for handling HTTP requests, persistence, authentication, downstream proxies, logging, alerting and extensibility. Acts as a man-in-the-middle between client and server, thus allowing the tester to intercept and modify all HTTP requests between both parties. (http://portswigger.net/suite/)

Read more…