What Do Software Testers Do, Exactly?

Well look, I already told you! I deal with the goddamn customers so the engineers don’t have to! I have people skills! I am good at dealing with people! Can’t you understand that? What the hell is wrong with you people? - Office Space

The role of testing has been a popular theme on this blog. We’ve written about a tester’s function extensively, published guest posts and asked our Testing the Limits guests their thoughts on the matter. Despite our best efforts however, the role of a software tester is still unclear to many.

Case in point: In a recent SlashDot thread someone asked the question “What does a software tester’s job constitute?” Here’s what he actually wrote:

“I got a call from a recruiter looking for software test engineer. I’m a software engineer and my job is development and testing. I know I mentioned testing but I’m pretty sure it’s totally different from professional testing practices. Can anyone shed light on what a software test engineer’s day to day responsibilities are? They said they’ll call me back for a screening and I want to be ready for it. Any tips?”

From there, about 200 people chimed in with their thoughts. Before I ask you to answer this question, I wanted to share some of my favorite responses from the thread. Note that not all of them answer the question directly (or correctly), but they are still valuable for the sake of discussion. Here are a few, in no particular order:

  • I am a developer and I tell my testers to consider me to be evil, lazy, and malicious. They must assume I am actively trying to fool them into thinking the application is working even if it is not with the minimum amount of work possible. That generally gets them to find the defects.
  • If you want to be a low-paid button pusher, yes. Do the same thing over and over, all day long with no deviation. If you want to enjoy your job testing, test the software. Try to break it. Troubleshoot. Do things the developers wouldn’t expect. (After all, who expects an apostrophe in a name field?)
  • The job of a software tester (tester, not button pusher) is to try to find all the defects in the software and report them to development so they can be fixed.

Continue Reading

Crowdsourcing Travels To A Galaxy Far Far Away

We already know that crowdsourcing is a great way to do software testing. And it proved pretty handy when creating a tribute music video. Now it turns out it even works for creating full-length films!

“Star Wars Uncut” is the full length Episode IV: A New Hope movie (running at 2 hours and 4 minutes), done in 15 second, crowdsourced clips. Crowdsourcing can produce some pretty amazing results.

If a Bug Cannot be Found, Does it Exist?

Deep thoughts today on the uTest Blog. Out of curiosity, I did a little research on the most difficult bugs to uncover and found an interesting answer: the Heisenbug. For those unfamiliar with this term, here’s a good definition from Wikipedia:

Heisenbug is a whimsical computer programming jargon term for a software bug that seems to disappear when one attempts to study it. The term is a pun on the name of Werner Heisenberg, the physicist who is commonly associated to the observer effect of quantum mechanics, which states that the act of observing a system inevitably alters its state.

How would such a bug manifest itself in the real world? Here’s one example:

Heisenbugs occur because common attempts to debug a program, such as inserting output statements or running it in a debugger, usually modify the code, change the memory addresses of variables and the timing of its execution.

One common example of a Heisenbug is a bug that appears when the program is compiled with an optimizing compiler, but not when the same program is compiled without optimization (as is often done for the purpose of examining it with a debugger). While debugging, values that an optimized program would normally keep in registers are often pushed to main memory. This may affect, for instance, the result of floating-point comparisons, since the value in memory may have smaller range and accuracy than the value in the register. In these cases, the bug is in the optimizing compiler and not in the program.

Of course, the idea of a disappearing software bug leads one to ask: if a software bug cannot be found, does it really exist?

Continue Reading

BugBuster: Play Our New Facebook Game for a Chance to Win an iPad2

You can kiss productivity goodbye for the next four weeks, because we have just launched BugBuster – a new gaming contest on Facebook!

Here are the details:

  • In order to participate, you must “like” the uTest Facebook page
  • Next, click on the Bugbuster game page, “Go to app,” and finally “Play” to begin
  • Contestants have until March 7th to participate (unlimited number of plays)
  • Awards will be given to the top 3 highest scorers:
    • 1st Place: iPad 2
    • 2nd Place: Digital camera
    • 3rd Place: Digital camera

We look forward to your participation in this contest and invite you to dethrone the current highest scorer! Remember, you have unlimited attempts until March 7. Winners will be announced shortly thereafter.

Happy bug catching!

13 Programmer Personalities You’ve Probably Dealt With

13 Programmer PersonalitiesWhether you’re on an in-house QA team or an independent tester, you’ll probably recognize these 13 distinct programmer personalities (compiled by PCWorld). Maybe you love some, maybe you hate others but they’re all out there and they probably won’t be going away anytime soon. So read on, and leave us a comment to let us know which type of programmer is your favorite from a testing standpoint.

1. The Underdocumenter
The Underdocumenter will go to any length to avoid being shackled by management’s foolish requirement to write text about that function. The droll ones will create functions like queryDatabase, then add a comment that says, “Queries database.”

Car: Vespa
Relationship status: Living with the same person for 15 years without getting married because they don’t want to fill out the forms
Household chore: Rewiring the house without labeling the breakers
Role model: Guy who hid the Ark of the Covenant before “Raiders of the Lost Ark”
Pet: “Around here somewhere.”
Favorite programming construct: Lambda
Drink: Anything with an “XXX” on the bottle

2. The CYA Specialist
If you’re lucky, your CYA Specialist will be a frustrated novelist who is happy to inject a pun or two into a boring pile of code. But the worst kind is the one who lords their documentation over others during code reviews. If a bug appears, the CYA Specialist says it was a limitation that was “well-documented in the 17th paragraph of the method’s comment.”

Car: Stack of Chilton manuals
Relationship status: Married to a 48-page prenuptial agreement
Household chore: Relabeling the spice rack
Role model: Wikipedia editor of the year
Pet: “Come over to see the photo montage of Scrappy that used to be just a wall.”
Favorite programming construct: The comment block
Drink: Triple-filtered water

Continue Reading

The Software Testing Mindset

Sometimes, work can be difficult without the proper mindset. If you’re tired, angry or frustrated for instance (like Patriots fans this morning) then you’re almost guaranteed to make some careless mistakes. This is true of almost every profession. Software testing in no exception.

So what’s the proper mindset for a software tester? Much has been written on the topic – and it’s critical to your success in the field – so I figured I’d offer a view different points of view.

Here’s one from softwareprojectmanager.com:

A professional tester approaches a product with the attitude that the product is already broken – it has defects and it is their job to discover them. They assume the product or system is inherently flawed and it is their job to ‘illuminate’ the flaws.

This approach is necessary in testing.

Designers and developers approach software with an optimism based on the assumption that the changes they make are the correct solution to a particular problem. But they are just that – assumptions.

Without being proved they are no more correct than guesses. Developers often overlook fundamental ambiguities in requirements in order to complete the project; or they fail to recognise them when they see them. Those ambiguities are then built into the code and represent a defect when compared to the end-user’s needs.

Continue Reading

Testing For All Occasions – Superbowl Edition

Sport Evac ScreenshotTesting usually means looking for bugs in software. But to Sport Evac, it means ensuring everyone gets out of a stadium safely in an emergency situation. The software was designed by the National Center for Sports Safety and Security and tests every scenario imaginable to help security personnel and first responders at big sporting events prepare for an emergency. But, as it turns out, testing is testing whether you’re testing the latest mobile app or an evacuation strategy for the big game. The goal is to find as many of the nasty bugs as you can before everything goes live. FoxNews has more on Sport Evac:

The Sport Evac program trains teams for those “what if” scenarios, by creating virtual 3D stadiums drawn from actual blueprints and packing them with up to 70,000 animated human avatars designed to respond to threats as unpredictably as their human counterparts.

It’s advanced technology fans won’t see on game day — but tech that behind the scenes makes watching the big game safer.

Evacuation in the event of an emergency is a critical challenge, and rehearsing the how to move around tens of thousands of people isn’t realistic. So sports security planners turn to computers to make sure it all goes smoothly.  …

The virtual stadium allows them to simulate how fans will respond in those first few critical minutes after an attack. Stadium and team security can use the virtual stadium to practice moving players and fans to safety and to run exercises with local first responders. …

Continue Reading

Classic Software Testing Mistakes

Every once and awhile, when there’s nothing topical to blog about, I decide to go back in time and focus on a software testing classic. Today is one of those days.

With that in mind, I wanted to draw your attention to Classic Testing Mistakes by Brian Marick. Whether you’re a tester or manager, experienced veteran or wide-eyed newbie, this 25-page article outlines some of the most classic testing mishaps and offers valuable tips on how to avoid them.

So what are some classic testing mistakes? One would be not reading this document in its entirety. As for the others, here are a few clips I found interesting. Note the bold titles are mine, everything in italics in Brian’s.

Mistake: Testers Not Responsible for Usability
If usability problems are not considered valid bugs, your project defines the testing task too narrowly. Testers are restricted to checking whether the product does what was intended, not whether what was intended is useful. Customers do not care about the distinction, and testers shouldn’t either.

Mistake: Misunderstanding the Role of “QA”
A first major mistake people make is thinking that the testing team is responsible for assuring quality. This role, often assigned to the first testing team in an organization, makes it the last defense, the barrier between the development team (accused of producing bad quality) and the customer (who must be protected from them). It’s characterized by a testing team (often called the “Quality Assurance Group”) that has formal authority to prevent shipment of the product. That in itself is a disheartening task: the testing team can’t improve quality, only enforce a minimal level. Worse, that authority is usually more apparent than real.

Mistake: Bad Timing on Load Testing
Putting stress and load testing off to the last minute is common, but it leaves you little time to do anything substantive when you discover your product doesn’t scale up to more than 12 users.

Continue Reading

Software Testing Slang

If you’ve worked in the software industry, chances are you’ve encountered words and phrases that you did not understand. I certainly have. So to help make sure we’re all on the same page (that means common understanding), I’ve decided to start a running thread of some of the more common slang terms that apply to software testing. Of course, this list will be woefully inadequate without your input, so please add your slang terms in the comment section below.

Here’s a few off the top of my head, as well as some that I found in this Quora thread:

  • Dogfooding: When a company tests its own software internally before releasing to beta
  • Low hanging fruit: Easy tasks that can be completed in short order
  • WAG: Wild ass guess
  • SWAG: Scientific wild ass guess
  • Staging: A development enviroment; one level before production
  • Automagically: Describes something that occurs in software that is either too complicated to explain, or the person describing the process really has no clue how it works
  • Quick and Dirty: A quick and simple solution to an otherwise complicated problem
  • Showstopper: A bug that makes software unusable
  • Brown-bagger: A very embarrassing bug found soon after release
  • Whack-a-mole: The practice of repeatedly getting rid of a bug, only to have it continually reappear
  • Drink the Kool-Aid: Lacking objectivity
  • Heavy lifting: Difficult and challenging work
  • SoLoMo: Social, location and mobile – testing components of many mobile apps
  • Burndown: a chart that is a graphical representation of work left to do versus time
  • FUBAR: F#@cked up beyond all recognition
  • PEBKAC: Problem exists between keyboard and chair (good one John Montgomery)
  • RTFM: Read the F#$cking manual
  • Fast-track: To speed up a process

Like I said, I’m certain that I’ve left out 99.99% of all the great testing slang terms. So please, pick up the slack (help me out) in the comment section below.

SQL Injections Still Top Threat

SQL Injection No. 1 ThreatGuess what? The No. 1 biggest security threat to your website is still SQL injections (not one of those hacker collectives that have been taking down websites left and right recently). SQL injections aren’t anything new – they’ve been on the Open Web Application Security Project‘s list of Top 10 threats since 2004 … when they started compiling the list.

Despite being on the security radar for literally years, injections still make up the vast majority of security issues today. From PCWorld (emphasis added):

SQL injection attacks have been around for more than ten years, and security professionals are more than capable of protecting against them; yet 97 percent of data breaches worldwide are still due to an SQL injection somewhere along the line, according to Neira Jones, head of payment security for Barclaycard.

Speaking at the Infosecurity Europe Press Conference in London last week, Jones said that hackers are taking advantage of businesses with inadequate and often outdated information security practices. …

In October 2011, for example, attackers planted malicious JavaScript on Microsoft’s ASP.Net platform. This caused the visitor’s browser to load an iframe with one of two remote sites. From there, the iframe attempted to plant malware on the visitor’s PC via a number of browser drive-by exploits.

Continue Reading