Software Testing Blog

Okay, so you beat Zelda blindfolded, reached the Donkey Kong kill screen on your first try and played Tetris for 14 consecutive hours. Big deal! If you really wanted to impress us, you’d have spent that time finding some bugs on level 1 instead.

As GamingUnion.com recently discussed, hard-core gamers do not necessarily make the best gaming testers. In light of  the “bugs, glitches and crashes” currently damaging the reputation of  Call of Duty, Battlefield and others popular titles, the article seeks to explain why gaming gurus are apparently “asleep at the wheel.”

There’s a lot of relevant content for software testers of all industries and verticals – so you should check out the article – but if you just want the gist of it, allow me to summarize their top findings:

Gamers Lack the Creativity of Testers
Of course, an expert gamer will be able to stroll through one level after another. That’s what expert gamers do. But what about novice players (like @matjohnston) who fumble around with the controller, claim that it’s broken, and can barely get past the title screen without dying? [Matt's rebuttal: first of all, that's not true... second, the computer cheats!]. A good tester will always keep the end user in mind, whether they’re a beginner or expert. They will test for actions that to them seem absurd (because they would never do such a thing), but that could in fact occur. Testers try to envision as many of these scenarios as possible. Apparently, many gamers do not.

Read more…

As expected, the iPad has received some terrific reviews following its debut to US consumers this past weekend. Users were quick to marvel at its sleek look and feel; its media capabilities; its usability and about a million other features.

Tech guru Matt Cutler (@mcutler) said “it makes your phone screen seem really small… and your laptop feel pretty dated.” Engadget writer Joshua Topolsky (@joshuatopolsky) called it a “potentially a prime mover in the world of consumer electronics” and Steven Colbert especially loves the way the iPad makes fresh salsa.

Is there anything it can’t do?

Of course. With the fanfare comes the feedback – both positive and negative – as evidence by the bugs, glitches and other inconsistencies that have since been reported. Here are a few of the more commonly referenced issues:

Read more…

Did you ever have that teacher who would give up on Friday afternoon and let the class watch a movie? We didn’t, hence your weekend reading assignment. What else are you gonna do while you wait in line for the iPad? That’s what I thought.

So, here are some testing/crowdsourcing articles to help pass the time. There will be a multiple choice exam on Monday. Or maybe we’ll just watch Ferris Bueller’s Day Off instead. Be prepared either way.

Serve Your Stakeholder
“Dr. Cem Kaner describes stakeholder as someone who has a vested interest in the success of the testing effort and/or in the success of the product. Stakeholders could be project managers, marketing managers, product managers, programmers, technical support representative, sales representatives, customers and many others in many different roles. Testing is done on behalf of the stakeholders. Here is a question for you. Have you ever asked who your stakeholder is before you start to test?” (From Parimala Shankaraiah’s latest post on curioustester.blogspot.com)

CNN Believes Whatever Computers Say
“Note to CNN: Information retrieved from an onboard computer is not reliable in cases where the computer itself is a suspect in the crime. No one is claiming that ghosts are causing the cars to go out of control. And there is no evidence yet that a mechanical failure is the culprit, despite the NHTSA looking hard for that evidence. Alcohol is also not a factor in these specific cases (or else we wouldn’t even be talking about them). That leaves two big possibilities: sober experienced people are suddenly forgetting how to drive OR something is wrong with the software or electronics that controls the cars.” (From James Bach’s blog on Satisfice.com)

Read more…

In becoming self-aware, modern software has developed an unusually cruel sense of humor.

Last week, for instance, a woman hit the jackpot at a Colorado casino – to the tune of $42 million – only to be told that her winnings were the result of a “software malfunction.” Of course, the casino refused to pay the full amount, but graciously offered to comp her room and drinks.

“It seems like a fair deal to me,” said the woman. “I could have really used that forty-two million, but free drinks and a hotel are just as good. I’m the happiest person in the world.”

Just kidding, she’s completely irate. As expected, the dispute has now entered the legal arena. Call me a cynic, but I think she’ll have a better chance of winning at the slots.

Meanwhile in Orlando, a guy making a routine bank transfer was shocked to see his balance at $88,888,888,888.88. According to the news report, he was not the only one to sneak into the billionaire’s club that day. He pondered moving the money offshore ( honest guy!) but before he could find the area code to the Cayman Islands, the issue had been identified and resolved.

Read more…

It’s back to the basics for the US Census. After several failed attempts to “modernize” the national headcount with hand held computers, officials are now blaming the resulting confusion, delays and costs on – wait for it – software bugs. You can read more about it here. To save you some time, I’ve summarized the progress made along the way with some telling quotations from those in the know:

Phase I: “Critical software errors are increasing; system performance is still lagging and testing continues to be compressed. A shortfall in testing portends potentially significant technical problems in the field.” – John Thompson, former associate director of the Census Bureau

Phase II: “The testing of the system is continuing to reveal critical defects.” – Todd Zinser, Commerce Department Inspector General

Phase III: “The performance of this system is not taking the load we’d like. It’s not going to accept the load that we will need to get it in about a month or so.” – Robert Groves, Census Director

Results: “They have prepared to do the follow-up using pencil and paper, and they’re very good at that.” – John Thompson (again)

Read more…

We usually like to keep things pretty light around here.  But this post is a public service announcement of the most urgent nature.  I don’t want to alarm anyone, and I’m not prone to exaggeration, but clearly software apps are rising up for the coming war against the humans.

First it was our cars (and then more cars); then it was unmanned aircraft.  But now, it’s gotten serious – because now the software uprising of 2010 is messing with our games.

Nick Saint (@ncsaint) over at Business Insider describes just how bad things have gotten in this latest battle between man and machine:

Owners of older models of Sony’s PS3 have been afflicted by a bug in the system’s internal clock. Unless you have a PS3 Slim, leave your machine off until word comes down that the bug has been fixed, or risk permanently losing data.

What’s next — our Foreman grills?  Our laser pointers?  Our lava lamps?  So consider this a call-to-arms for all who develop and test software.  The war is on.  And lately, the software (and its well-hidden bugs) are winning.  Izzy Mandelbaum was right:  It’s go time here, people!

Retired NASA Astronaut Mike Mullane* (pictured left) said it best when he asked: “Why is there never time to do it right, but always time to do it over?” He could have easily been talking about the recent problems Toyota has been dealing with, but he wasn’t. He was talking about today’s software companies.

Conversely, this recent article from The Economist could just as well be about today’s software companies, but it isn’t. It is about Toyota’s recent problems.

Like everyone else, the author wants to know how the auto giant could so quickly lose its reputation for safety and quality (things that can happen to ANY company if they are not careful). The culprit? You guessed it: software bugs.

Instead (of trying to keep pace with competitors), two recent trends, both software related, hint at the reason behind Toyota’s unexpected decline. One is the shortening of product-development cycles generally in the car industry. These are down from a typical four or five years to little more than 15 months, thanks to computer-aided design and manufacturing, and the virtual simulation of the resulting products. To save money and time, Toyota has even dispensed on occasion with building test “mules” and other engineering prototypes.

Read more…

I can’t take credit for the pun in the title (it’s from The Simpsons’ Treehouse of Horror*) but it fit so well with today’s theme, I just couldn’t help myself.

That’s right, it’s Halloweekend (that lame pun is mine). And so despite the warnings from that old guy in Pet Sematary who said that “Sometimes…dead is better,” we’ve decided to help a few scary software bugs rise from their graves. What could possibly go wrong?

Grills to Cook Babies and More

From OrigSoft.com: “It’s a well-known problem with websites that if you trust user-submitted data that you will get burned. Sears literally did get burned by their own incompetence when their website started promoting ‘Grills to cook babies and more’. The problem wasn’t a huge lack judgment by the Sears product team, but rather a lack of understanding about displaying variable names and values in the URL. A lot of sites do this by default, but the Sears site took it one step further. If a specific page became popular, the results were cached and displayed to users.”

The Ping of Death

Read more…

Remember that scene in Office Space where Michael Bolton (no, not him) realized that he’d botched The Plan?

If you recall, he and his accomplices developed a computer virus that was supposed to deposit pennies from each company transaction into their own private accounts. This money would of course grow at an exponential yet unnoticeable rate, thus securing them with a cozy retirement fund. No more work. End of story. Roll the credits.

Nope. Instead, a software bug transferred hundreds of thousands of dollars to their account…instantly. Suddenly, a comfortable retirement (i.e. “I would do nothing”) was replaced by the prospect of federal prison. You know how the rest of the story goes, but here’s the lesson: If you plan on stealing a lot of money through software, test it out first. If your scheme works, then try to keep a low profile.

These guys did not get that lesson. Either that, or they’re the luckiest gamblers in North Strabane, Pennsylvania.

Read more…