Software Testing Blog

Ho, ho, ho!  Whoa there, Blitzen– wasn’t it just Halloween?  It sure feels that way. After all, I still have two pounds of trick-or-treat candy to pretend I’m not eating.

Unfortunately, my four-year-old has already implored me to take down the skeleton and spiders hanging in the doorway because they’re going to scare away Santa.  So, rather than arguing the salient fact that Santa shimmies down the chimney versus ringing the doorbell, I’ve officially started morphing decor from the marvelous macabre to merry old Saint Nick.  Kids: 1. Mom: 0.

Nonetheless, the fact hasn’t escaped me that we’re two weeks away from Cyber Monday (November 28th), an occasion that online retailers have been planning for months.  Since summer, global brands and independent e-tailers have been testing and re-testing their mobile apps and web sites for functionality, usability, localization glitches and possible bottlenecks in site performance that could jeopardize their revenue potential.

Moreover, the ante has been upped now that the iPad and other tablets have entered the scene.  Online retailers that spent the last few years optimizing their mobile apps and porting them to additional platforms like Android, are now going through the process from scratch with tablets.  Not only are the specs non- standardized, varying significantly by manufacturer, device and network performance like smartphones.

Read more…

It’s not fair, but it’s true: Stereotypes exist in every profession. Software testing is no exception. We’ve covered this topic before, so why bring it up again? Good question. I recently came across a great article by Modha Khammammettu on softwaretestpro.com on how to become a less stereotypical tester. First, his definition the tester stereotype:

A stereotypical description of an application tester’s role is usually about being given specifications, reporting bugs, verifying and closing bugs. This stereotypical tester will have a part to play in a testing organization, but will always be considered an outsider to the project and will not have much impact on the project lifecycle.

If we step out of the boundaries and terminology used in testing, a test engineer is valuable to the project when he/she helps steer the project clear of pitfalls during the course of the project.

Sound familiar? He then lists some great, actionable examples on how to become more effective in the realm of deployment topology, design and coding standards, project methodology and feedback, of which he writes:

One accurate measurement is worth a thousand expert opinions. For instance, it is always better to say that a particular page takes 35 seconds to load than saying that the page is ‘slow’. In case a fix is made, we’ll be able to easily compare the load times to check if the fix really addressed the issue.

Give the developer the exact scenario for the failure. My experience has shown that providing a script that a developer can run to reproduce the issue saves time compared to describing the bug in a wordy manner. Sometimes I just record the steps using a screen recorder. Providing insight into why a failure may have occurred also helps the development team.

Of course, there are many stereotypes in the field of testing. Here are a few more courtesy of testinggeek.com:

Read more…

On Wednesday, we saw the first ever nationwide test of the Emergency Alert System. Well, I didn’t actually see it myself – I was on the web at the time – and neither did about 10 to 20% of TV and radio audiences. But everyone else…

As you might expect, many news outlets are calling the test a failure, because it uncovered so many flaws. If you’re a professional tester, you should laugh at the irony of this statement. I did.

Anyway, here are some details from CNN.com:

Problems were reported across the country during the first-ever nationwide test Wednesday of the Emergency Alert System, designed to allow the president to address the American people during a national emergency.

The Federal Emergency Management Agency and the Federal Communications Commission, which ordered the test, stressed that it was designed to find flaws, and scoffed at reports the system had failed.

By late Wednesday afternoon, an FCC official, not authorized to speak on the record, said about one-third of the test participants had filed preliminary reports, and those showed that 80% to 90% of the stations received the alert and were able to rebroadcast it, which was the major criteria of the test.

The official called the failure rate of more than 10% “not insignificant,” but said identifying problems “is why we have the test.”

The FCC says it will not release specific test data to the public because broadcasters worry that potentially embarrassing results could discourage participation in future tests, and test data could reveal security vulnerabilities.

Earlier in the post I mentioned the fact that I was on the web (i.e. not watching TV or listening to the radio) and was left out of the emergency test. If you think this could leave a large percentage of the population in the dark, you’re not alone. As it turns out, FEMA and DHS are working on a system of mobile and web alerts.  CBSnews.com has an update:

Read more…

Game Over?

In the battle over the mobile web, the Flash Mobile Player has officially been blocked by the HTML5 lineup (arguably with Steve Jobs as the forward-thinking QB). In an Adobe blog post yesterday, VP Danny Winokur stated:

“We will no longer continue to develop Flash Player in the browser to work with new mobile device configurations (chipset, browser, OS version, etc.) following the upcoming release of Flash Player 11.1 for Android and BlackBerry PlayBook.”

For phones and tablets, the future is clear; however, the battle isn’t over. According to Mashable, “Adobe has added more robust cross-platform mobile development features to Flash Professional and added native iOS streaming to Flash Media Server,” maintaining “a strong commitment to Flash as a development platform separate from a technology stack.”

In line with more brands moving toward a hybrid approach (see post on Pandora), Adobe is astutely refocusing its efforts on native apps and aggressively contributing to HTML5.

What do you think? Without mobile, has Adobe Flash become irrelevant?

Update: Adobe also told GigaOM it has stopped supporting Flash on digital home devices, such as HDTVs.

From the people who brought you uTest on Twitter, Facebook, LinkedIn, Youtube and Flickr…..

That’s right, uTest is now on Google+!

In case you missed it, Google+ recently opened things up to brands and businesses. Some companies have been slow to make their presence felt on the new social media platform, but not us.

However, since our Google page is less than 24 hours old, we’re noticeably short on followers. So if you’ve got a minute to spare – and want to keep up on the latest testing trends – check out our Google+ page and make the connection (actually, it’s called circling). You’ll be glad you did.

In the meantime, if you’re thinking about creating a Google+ page for your own business, here are some great tips from Google, who else?

We’ve spent a considerable amount of time and energy proselytizing the masses on need for in-the-wild testing (i.e. testing applications with real testers, in real-world environments). However, we’re certainly not the only ones on board with this philosophy.

Here’s an interesting story from The Ottawa Citizen on what could possibly be the world’s most interesting regression test:

Taking off from the flightline here Oct. 27, a B-2 Spirit travelled to the top of the world and back again on a mission to test the aircraft’s hardware and software upgrades, endurance and performance at extremely high latitudes.

The more than 18-hour mission to the North Pole and back to Edwards AFB consisted of developmental and operational test points to prove that the B-2′s software upgrade works well and is able to operate anywhere in the world.

Although the B-2 has been to the North Pole in simulated tests, this is the first time the aircraft has physically travelled there, making this a milestone in B-2 testing.

“A goal of the test force is to prevent a situation where an aircraft experiences an anomaly with a new system for the first time in an operational mission,” said Lt. Col. Hans Miller, the 419th Flight Test Squadron commander. “This flight to the North Pole could reveal data and lessons that were not seen in a lab or simulated environment.”

Sound familiar? Of course, the main reason why in-the-wild testing is gaining ground in the software space include factors such as:

Read more…

If you asked 100 software testers what traits translate into success, somewhere near the top of that list you would find “creativity.” But what makes a tester creative? Is it a skill to be learned? Is one born with it? Well, according to a recent study, the answer might have something to do with video games.

A test of 491 12-year-olds found that the more they played video games, they more creative they were. The research by Linda A Jackson, a Professor of Psychology at Michigan State University, has been published in current issue of journal Computers in Human Behavior.

Playing with cell phones, the Internet and computers (other than for video games) was unrelated to creativity, the study found.

Jackson’s team assessed how long the kids spent on different types of technological activities and then tested them with the Torrance Test of Creative Thinking to measure creative output, tallying up the two pieces of data. Torrance tests involve tasks such as drawing an “interesting and exciting” picture from a curved shape, giving the picture a title and then writing a story about it. Testees score better if their responses are unusual, infrequent, or elaborate.

Of course, video games aren’t the only way testers can become more creative. In fact, uTester Amit Kulkarni offered testers some great advice on being a more creative tester in his last guest post. Here’s a nice extract:

Read more…

Does it make a sound? With more than 500,000 apps in Apple’s app store and more than 200,000 apps in the Android  Market, I’ve often wondered, when a new app drops (and no one is around to hear it), how can it make enough noise to attract users?

Start with focusing on a particular market segment, says ReadWriteWeb. RWW published a very interesting post today to give mobile developers some insight into what they should be thinking about before building their killer app.

The main gist of it was to focus more on whom you’re developing for vs. the functionality of the app. I’m guessing this is the part where testers all over the world want to beat someone up right about now. Of course testers want developers to develop with functionality in mind; however, I do think the article brings up a good point.

Balancing the technical side (functionality) with the business side (target audience) will get you one step closer to having your new app “heard” and raking in the cash.

Read more…

Of course, as soon as the iPhone enters the wild it seems like a flood of bugs emerge.  As of earlier this week there were already close to 100 iPhone 4S’ in the community (and growing rapidly) so I’m curious to hear what else our testers have seen.

The most recent complaint is the battery drain, but there are others, as documented in this YouTube video and listed below.

• iPhone 4S Battery bug
• Audio Bug

Add your iOS 5 bug list in the comments below.

P.S. – Apple, did you know we’re giving away free test cycles?

We’ve all seen them. We’ve all used them. And we’ve all learned a new word or two in the process. I’m talking of course about captchas, which are used to ensure that responses are generated by a person, not an automated script. The most popular use case involves (I think) changing passwords.

Anyway, if you’ve ever doubted the effectiveness of captchas, you’re certainly not alone. Here’s a great story from CNet.com on captcha fail:

A team of Stanford University researchers has bad news to report about captchas, those often unreadable, always annoying distorted letters that you’re required to type in at many a Web site to prove that you’re really a human.

Many Captchas don’t work well at all. More precisely, the researchers invented a standard way to decode those irksome letters and numbers found in Captchas on many major Web sites, including Visa’s Authorize.net, Blizzard, eBay, and Wikipedia.

Their decoding technique borrows concepts from the field of machine vision, which has developed techniques to control robots by removing noise from images and detecting shapes. The Stanford tool, called Decaptcha, uses these algorithms to clean up the image so it can be split into more readily recognized letters and numbers.

What do you suppose could be the main reason why captchas have been less than effective? You guessed it:

Read more…