Software Testing Blog

It’s been named the “Future Attribute Screening Technology” program – or “FAST” for short – and its purpose is to someday be able to predict a crime before it happens. The Department of Homeland Security has been testing FAST – which is said to factor things like body language, vocal pitch, ethnicity, gender, eye moments, body temperature and other variables – with government employees in an undisclosed location. You can read all about it on The Telegraph, among other news outlets.

Since this story involves a significant testing component, we really don’t need to mention…

The obvious comparisons to Minority Report: True, the FAST technology would probably never have been conceived if not for the sci-fi classic by Phillip K. Dick, who must be rolling in his grave right now. And if not for the absence of those creepy spider-like eye scanners, Steven Spielberg and Tom Cruise should probably demand a healthy royalty check if this program is ever rolled out for real.

The potential abuse of civil liberties: No, you’re not being paranoid if you see how this technology could somehow, someway compromise our rights to privacy and that whole innocent-until-proven-guilty thing. Not to worry though, as the DHS issued the following statement to calm such fears:

“The FAST program is entirely voluntary and does not store any personally-identifiable information (PII) from participants once the experiment is completed. The system is not designed to capture or store PII. Any information that is gathered is stored under an anonymous identifier and is only available to DHS as aggregated performance data. It is only used for laboratory protocol as we are doing research and development. It is gathered when people sign up as volunteers, not by the FAST system. If it were ever to be deployed, there would be no PII captured from people going through the system.”

Feel better now? Good, then we can move on to the testing aspect of this story without discussing…

Read more…

Sometimes there are features that just can’t be tested until the product is launched. For example, today I had a conversation with one of the uTest product managers about some upcoming features, and she lamented that there was this one thing she just couldn’t get tested. It’s an internal feature that changes the behavior for emailing our project managers, but testing it is nearly impossible because email is disabled on our staging server. Enabling emails would mean sending garbage messages to 35,000 testers, so this one feature won’t be tested until the last minute when it’s ready for launch. It’s not ideal, but it is understandable.

Another example: Robert Scoble recently interviewed Trey Ratcliff (one of my favorite photographers) about his new photo editing app for the iPad. Trey remarked that because there is no camera on the iPad 1, they had to “blindly” add the feature for taking pictures using the built-in camera – that is, without testing. That’s because when they wrote the feature, the iPad 2 (which includes a camera) didn’t yet exist. Kind of scary, but understandable. No amount of testing in the world would validate that feature until the iPad 2 went on sale.

The sad reality is that there are times when getting 100% testing coverage is impractical. Fortunately, there are some strategies we can use to mitigate any problems that may come up:

Read more…

We’re so in awe of all the amazing things our terrific customers have been able to accomplish with the uTest community that we think it’s definitely time to give them some serious ink. Check out VisibleGains’ story below (or on our case studies page)!

There was a time when customized video was reserved for the big players with big marketing budgets. Today, that’s not the case. VisibleGains helps companies of all sizes, from all over the globe, produce/track custom videos with a unique online platform.

But before this web app could be released into public beta, it needed testing across all major browsers and OSs. For too long Co-founder/VP of Product Craig Daniel was the lone tester and knew a more scalable solution was in order. “I thought this would be difficult considering it involved a lot of moving parts, but uTest handled it perfectly,” said Craig.

Less than 24 hours after opening the test project, Craig was able to review bugs and feedback from 35+ professional testers, who had submitted nearly 120 unique bugs! Find out more in the 2-minute clip below about Craig’s overall experience; how uTest gave him access to rockstar testers; and a showstopper uTest found!

VisibleGains Customer Testimonial

When we relaunched our testing platform last year, we chose to build it in Flex. It allowed us to build a nice web UI, but it also meant that iPhone and iPad owners couldn’t connect to our platform directly. An Apple iOS user had to find a regular computer to report bugs when testing an iOS app.

Well we’ve heard their pleas, and we’re happy to announce something that should help: the uTest iOS app. With a native interface for both the iPhone and the iPad, it’s now possible for testers and customers to test on the couch and on the go.

If that’s enough to make you want to download the app right now, then don’t let me stand in your way. Just click (or tap) on that image to the left and go get it. It’s free, you know.

If you’re still wondering what makes our app special, let me tell you about some of the awesome new things that both testers and customers can do.

For Testers

Customers and testers can start testing with the uTest iPhone app.

iPad users have a native interface that makes full use of the iPad layout.

It goes without saying that the uTest iOS app lets you participate in test cycles and that our app makes it easy to submit bugs right from your iPhone or iPad. But what’s really cool is that if you’re testing another iOS app, you can submit screenshots and videos of your bugs directly from the uTest app. You can even use your camera to take pictures for upload – handy if you need a screenshot of a bug on another mobile device.

In addition to all that, you can do all the other things you would expect while testing, like view the bugs submitted by other testers, reply to tester messenger conversations, and even check out your uTest earnings.

Of course, all this assumes you’re already a uTester. Because if you’re not, you can actually signup for a tester account right there in the app. It will even help you setup your iPhone or iPad as your first testing device on the uTest platform.

For Customers
If you’re a uTest customer, you’re going to love the uTest app. Why approve and reject bugs from a boring old computer when you can do it from the beach? And if you don’t have a beach nearby, how about the comfort of your couch? In fact, you can now review your test cycles from anywhere you like (assuming there’s a phone or wireless signal, of course).

With the uTest app, you’ll also be able to review attachments and even ask testers questions with tester messenger. Everything you need to keep an eye on a test cycle is available at your fingertips.

Wrapup
Of course, our community rigorously tested our iOS app and they discovered over 60 bugs before launch. Their diligence made this app super solid, and that helped us to get approved by Apple for the App Store in record time without having to resubmit.

Now that we’ve launched our first iOS app, we’re hardly finished. We want your feedback and ideas about how we can make it even better. uTest community members can join our tester forums and check out our Platform Feedback section. Customers can contact their project manager directly or drop us a line.

Our guest blogger this month is Atul Angra. A resident of India, Atul is one of our more accomplished testers (a Gold Tester in fact), with over six years of professional experience. He’s a photographer at heart, but a tester by trade, with domain expertise in healthcare and finance. He’s also a former Bug Battle winner, a guest judge, a Tester of the Year, a Forums junkie, a crash course author and he’s here today to discuss how interpersonal skills can make or break a tester’s career. Enjoy!

*******

Let’s take a scenario where a tester follows the rules and reports 100 bugs. Some of these bugs were traced to non-documented requirements that are implicit in nature, such as a drop-down list not populating alphabetically and things of that nature. These bugs are quite common and usually end up in conflict, as development teams reject them based on the argument that it’s not a defined requirement.

Here, both the developer and tester are not ready to close this issue – and they are both correct. The traditional way these issues are resolved is by involving someone from management to intervene and make a decision. The time spent in escalation and argument is much greater than what it would have taken to actually fix the issue.

At a high level, we could blame the team which collected requirement, but this may not be the case when it comes to implicit requirements. Many of these situations could be resolved if the tester demonstrates interpersonal skills.

Read more…

In this month’s installment of This Week In Testing, the date was May 1996 and the setting was the First National Bank of Chicago (insert dramatic pause here). The gist? Software “glitches” caused the bank accounts of 823 customers of the major US bank to be credited with a total of $924,844,208.32 each.

According to The American Bankers Association, all of $763.9 billion — more than six times the total assets of First Chicago NBD Corp. — was the largest error in US banking history.

And the reason given? Inadequate testing of course! The bank updated its ATM transaction software with new message codes. The message codes were unfortunately not tested on all ATM protocols, which resulted in some ATMs interpreting the codes as huge increases to customer balances.

This isn’t the first time we bring up banking bugs. You might remember Software Bugs: You Win Sum, You Lose Sum, the post about a man in Orlando who while making a routine bank transfer was shocked to see his balance at $88,888,888,888.88.

What other bugs have you recently heard or read about with such huge financial implications? Any mobile banking bugs?

In the past year, we’ve made tremendous strides in our march to change the world of software testing.  We started by doing functional testing for web apps.  Then came desktop apps.  And then mobile apps.  More recently, we’ve expanded to provide load testing services.

Since we clearly have nothing left to prove in the software testing game, we’ve set our sights on other testing markets that are in desperate need of our special brand of crowd-driven innovation.  And after months of market research (mostly just watching movies), focus groups (a collection of bobblehead dolls we bought on eBay), and with the help of the brilliant consultants at Prestige Worldwide (NSFW), we’re pleased to announce our revolutionary new testing services in the following categories:

Driving tests:  Can’t keep it between the lines?  Don’t know how to parallel park?  Not even sure what “10 & 2″ means?  No problem.  With a community of 24,000+ testers from 162 countries around the world, we have people who know how to drive on the left- or right-hand side of the road.  We can help with the written exam and the road test (Please note: this would be considered 2 separate test projects).

Food tests:  Afraid someone’s trying to poison you?  Think someone may have slipped something into your drink?  Or maybe you’re just concerned that your soup is too hot?  Taking unnecessary risks with your food is now a thing of the past.  Just ping uTest and we’ll have a tester on-site in minutes, ready to sample your food or beverage and deal with the consequences.

Paternity tests:  Not sure what your responsibilities are?  Scared to learn the results?  With this latest service, you don’t have to be.  We’ve got testers everywhere and we’re happy to help you pass the test of a lifetime.  A member of our community can either take the test for you or help you study for it yourself.

Read more…

With hundreds of thousands of iPads being pre-ordered; and with HP releasing its Slate this year; AND with German company, Neofonie announcing the WePad (running on Android), the tablet market is definitely opening up some unique opportunities for the testing landscape.

According to Flurry Analytics, nearly half of the apps being tested on the iPad fall into the games category — a  whopping 44% of Apple test time.

On the other end of the iPad app testing spectrum (only 3% of app testing falls into the ‘books’ category) but highly anticipated, E-reader companies like Amazon and Barnes & Noble are eagerly preparing their iPad apps (see iPad Kindle reader sneak here), gearing up to go head-to-head with Apple’s bookstore.

With the race on to build the first iPad apps, what are the risks of not being able to yet own or hold one? The New York Times reports:

“neither company [Amazon or Barnes & Noble] was given an iPad for testing” and “there are real-world factors that may go undetected with a simulator, like the weight of the device and how people hold it.”

As we all know here around uTest, there’s a world of difference between on-device testing and testing in a simulated environment. And with mobile app testing still maturing as a discipline, what challenges (or opportunities) will iPad, WePad and Slate apps bring to the world of testing?

QUEST, one of the top software testing conferences, will be held in Dallas this year (April 19-23).  And uTest is getting geared up and is thrilled to be a part of this conference.

In addition to inviting Doron to be a keynote presenter, QUEST features a week-long agenda packed with more than 100 opportunities for attendees to build new skills and prepare for the testing professions of the future.

From exploratory testing to test automation to security audits to crowdsourced testing,  QUEST will cover a wide range of testing topics that give attendees insight into the latest best practices and innovative approaches to testing today. To learn more, here’s a sneak peek at the QUEST Magazine.

Special Note: Members of the uTest community interested in registering for QUEST are eligible for

Read more…

With our testing community currently hammering away in the “Bug Battle of the TV Networks” this week, it’s time to take a moment to reflect on our February bug-iversary.

On February 11, 2007, during its very first overseas deployment to Okinawa, Japan, six F-22 Raptors flying from Hawaii experienced multiple computer crashes, including navigation, communication and fuel system crashes, when crossing the International Date Line.

Read more…