Software Testing Blog

You know how they say a picture is worth a thousand words? Well in the case of Android fragmentation – a subject discussed at great length here and on mobileapptesting.com – a picture is worth a few thousand devices.

The image you’re seeing is a data chart of thousands of separate device models encountered by the dev team at OpenSignalMaps over the course of a six month period. It may not be a flashy infographic, but it’s one of the best visuals I’ve seen to convey the challenge of Android hardware fragmentation. Here with more details on the chart (and on Android fragmentation in general) is arstechnica:

Read more…

Kickstarter recently experienced a security bug that made information from 70,00 unpublished projects accessible through the API connected to their homepage. The problem? Kickstarter’s new homepage didn’t play nice with the API. From Kickstarter’s blog:

The bug was introduced when we launched the API in conjunction with our new homepage on April 24, and was live until it was discovered and fixed on Friday, May 11, at 1:42pm. The bug made accessible the project description, goal, duration, rewards, video, image, location, category, and user name for unlaunched projects. No account or financial data was made accessible.

Luckily, the majority of the breached information was accessed by a Wall Street Journal reporter who notified Kickstarter about the issue. While information from 70,000 projects was accessible, only 48 projects were accessed by people other than the reporter.

Now I don’t know Kickstarter’s testing practices, and maybe this was just one of those things that unfortunately slipped through the cracks, but it seems like a prime example of the need for regression testing. When you’re introducing a new element to your site, or software or app it’s extremely important to make sure it doesn’t break any existing components – especially security related aspects. Kickstarter’s lucky personal or financial data wasn’t leaked, but that may not be the case next time. So let this be a lesson on why regression testing is so important!

If you know a thing or two about Unix, then you know to be careful with the ‘rm’ command. Those two letters – short for the word ‘remove’ – can delete files faster than you can think twice, shout “no, stop!” and hit CTRL-C.

Pixar had something very similar happen to them when they were developing their hit movie Toy Story 2. A mistyped rm command inadvertently deleted the entire movie. Then, to make matters worse, they discovered their backups were all bad. This short video, narrated by two of the technical leads from Pixar, explains how the whole thing went down and the one random thing that saved the whole project.

A good reminder to everyone: whether you’re developing the next great app or producing the next blockbuster movie, backup often and make sure your backups actually work.

When it comes to finding defects in web applications, timing is everything. A bug found in production, for instance, will generally complicate matters more than a bug found in pre-production. This is also true of uTest projects, when testing is sometimes compressed into a shorter (than ideal) time-frame.

So to help you find the right bugs at the right time, we suggest checking out the latest article from our old friend Matt Heusser. In Seven Ways to Find Software Defects Before They Hit Production, Matt shares some valuable tips on ways to reinvigorate your testing. Here are three techniques that I think uTesters will find especially useful. Enjoy!

Technique 1: Quick Attacks

If you have little or no prior knowledge of a system, you don’t know its requirements, so formal techniques to transform the requirements into tests won’t help. Instead, you might attack the system, looking to send it into a state of panic by filling in the wrong thing.

If a field is required, leave it blank. If the user interface implies a workflow, try to take a different route. If the input field is clearly supposed to be a number, try typing a word, or try typing a number too large for the system to handle. If you must use numbers, figure out whether the system expects a whole number (an integer), and use a decimal-point number instead. If you must use words, try using the CharMap application in Windows (Start > Run > charmap) and select some special characters that are outside the standard characters accessed directly by keys on your keyboard.

Technique 3: Common Failure Modes

Remember when the Web was young and you tried to order a book or something from a website, but nothing seemed to happen? You clicked the order button again. If you were lucky, two books showed up in your shopping cart; if you were unlucky, they showed up on your doorstep. That failure mode was a kind of common problem—one that happened a lot, and we learned to test for it. Eventually, programmers got wise and improved their code, and this attack became less effective, but it points to something: Platforms often have the same bug coming up again and again.

Read more…

As many of you know, uTest wouldn’t be where it is today without the innovativeness (yes, I just made up that word) of our testers, partners, investors and employees.

Of course, we already know this, so it’s extra special when a third party recognizes uTest for it’s innovative qualities. Last night – when uTest was named a 2012 Innovator of the Year by the Smaller Business Association of New England – was a classic example.

uTest was one of 92 finalists for the award, given to companies ”that are not merely successful, but whose success is due to innovation and thought leadership” and who show “explosive growth potential.” With that criteria, were we surprised by the win? Not really. Were we honored by the recognition? Absolutely.

The award puts uTest into good company (pardon the pun) with past winners, including Communispace, Genzyme, iRobot, KRONOS, Staples, VistaPrint, ZipCar and others. That’s quite the  list of entrepreneurial companies who were successful in disrupting their industries, and we were stoked to be included among such pioneers.

Even if we hadn’t won, it still would have been a great night. We had fun mingling with other attendees from the likes of HubSpot, Verivo Software, The Switch, LevelUp, MassHighTech, The Boston Business Journal and others. There was also a great keynote by David Friend, Chairman and CEO of Carbonite. All in all, a terrific night.

Once again, thanks to SBANE for the honor and thanks to uTest nation! We couldn’t have done it without you!

There has been plenty of talk these days about security. The increased use of computers and mobile devices to bank, shop and communicate with friends and family has also increased a user’s vulnerability to cybercriminals. Software updates and patches are critical to keep computers secure, but some companies are having trouble getting those patches released quickly.

Like many others, Apple recently had trouble with an exploited vulnerability, when cybercriminals were exploiting a flaw in Oracle’s java application environment. While Oracle was able to release an update for Window’s and Linux rather quickly, Apple (who handles their own Java updates) took months. There was much disappointment about the delayed response from Apple, and while many chalk this up to the fact that Apple was unprepared, as Mac’s had been virtually impenetrable for years, several others also cite regression testing as a major delay in releasing critical updates and patches. According to Sue Marquette Poremba of Security News Daily:

A quick fix isn’t always a good fix.

 ”Updating software reliably does not only mean fixing the problem,” (Wolfgang Kandek, chief technology officer at Redwood Shores) said, “but also testing whether the fix plays well with other modifications included in the code, plus making sure that it does not break any functions of the software.

Having a fix that works is important, but having the vulnerability on your computer affects how your system runs and who can control it. More often than not, the average computer user has no idea that a risk is there.

Read the full article at MSNBC.com >>>

Ask any of the 55,000+ testers at uTest and they can certainly tell you that bug fixes take time. To triage, locate and correct a bug is not an easy process by any means. And more importantly testing to make sure the fix is not going to break other existing features or open the application to additional vulnerabilities is critical.

We’d like to know how you feel about security updates. How long do you think security patches and updates should take?

It’s been a busy month for the uTest travel team. First there was STAREAST, then QUEST and now CTIA, one of the world’s largest conferences for the mobile & wireless industry. Basically, if there was an event in all CAPS, chances are uTest was there.

Not that you could tell from the picture, but uTest is actually well-represented down in New Orleans, where several employees were sent to mingle with the attendees and to see what’s new and what’s next in the world of mobile (check out the uTest Flickr stream for some great pictures of the event).

Note from Erica, one of our proud uTest representatives at CTIA: We’re bringing the show to you up-to-the moment on our Twitter stream.  But we also urge you to follow the official hashtag of #CTIA12 to get the scoop from other exhibitors and media, too.

Anyway, events like this are always a good time for companies to make headlines. This year has been no exception thus far. Here were four recent stories from CTIA that caught our attention:

1. Mozilla CEO Touts HTML5: Gary Kovacs, CEO of the company that makes the Mozilla Firefox web browser, encouraged his audience today to support HTML 5 as a way to insure that users will have a world of choice when it comes to mobile apps. “It is impossible for me to believe that one or two companies will be able to curate all the content,” said Kovacs. “Imagine the explosion of innovation if we could unload 5 or 6 million developers onto the ecosystem that is the web.” Read more…
2. New Mobile Devices: The new Samsung Galaxy S III and the HTC Evo V 4G made all the early headlines, but new waterproof devices from Kyocera (remember them?) as well as Clarity (who make smartphones for seniors) also caught the media’s attention. Read more…
3. Apple? Amazon? Bueller?: That’s right, Amazon and Apple – two mainstays of the mobile industry – decided not to attend this year’s event. Ina Fried discusses their absence and what it means for rival tablet makers. Read more…
4. More Data Please: Top brass from Verizon and T-Mobile said that “the future of data use, such as streaming video and photos, is at risk if more airwaves, or spectrum, aren’t put to use.” Read more…

What other stories are you following from CTIA? Be sure to let us know in the comment section.

Journalist Jeff Howe defined crowdsourcing as “The act of taking a job traditionally performed by a designated agent (usually an employee) and outsourcing it to an undefined, generally large group of people in the form of an open call.” And what’s the best way to get people to willingly respond to that open call and perform a (at times grueling) task? Make it a game.

And, as unlikely as it sounds, that’s exactly what medical researchers have done. It turns out that: A. people are very good at finding patterns and coming up with different ways to solve puzzles and B. two minds are better than one, and there are a lot of minds on the internet. That’s why games like Phylo and Biogames’ Telpathology game are growing in popularity while helping researchers solve critical problems.

Phylo is a web-based game that “harnesses the computing power of mankind to solve a common problem: Multiple Sequence Alignments.” Here’s what Phylo has players doing and how it’s helping researchers identify potentially lifesaving DNA patterns:

Read more…

A former co-worker of mine once worked for a prominent job-search site. I won’t disclose which one, but let’s just say they were a monster in the industry (wink wink). Anyway, she told me that traffic to the site peaked first thing on Monday morning, when disgruntled employees would return from a nice weekend back to a job they loathed. Some would call this a case of the Mondays.

This being Monday morning, chances are there are a few testers out there considering whether or not to quit. To help make that decision a little easier, I wanted to share this advice from a recent Mashable piece, 3 Signs That It’s Time To Quit Your Job. Since it was written for a general audience, I’ve inserted some testing-specific commentary.

1. Your Values Do Not Align With The Company’s Values

It is stressful to be in a situation when you are asked to carry out practices that contradict a company’s written policies or stated values, or that conflict with your own personal values. The cognitive dissonance alone can drive an employee up the wall.

You may decide that the reasons for leaving are mainly because your personal values do not fit with an organization’s values (even if at one time they were aligned), or you may decide to leave because an ​​organization does not even live up to its own values and contradicts its own written rules of professional conduct. Figure out what values are most important to you and what brings you job satisfaction.

In the tester world, I suppose one example of this could be the adoption (or lack thereof) of the agile methodology. A company or department saying they value the agile method is different from actually adhering to its principles. This has been called “fake agile” by Elisabeth Hendrickson and it can make dev and testing teams quite dysfunctional. Bottom line: As a tester, if you’re unable to influence key decision-makers about the benefits of a particular approach – and it’s hampering your ability to do your job - then it may be time to leave.

Read more…

As QUEST 2012 wraps up our intrepid videographer Stanton Champion snagged one more interview to keep the rest of us non conference goers up-to-date.

This time Stanton checks in with Anaf Durrani, of Orbitz, and gets a recap of Anaf’s standing-room-only presentation on mobile testing and continuous integration.

Orbitz wanted to be on the forefront of the mobile trend and realized they needed to refine their testing strategy. Anaf discussed how Orbitz went about choosing the tools they use and how they decided on the company’s unique combination of testing. Here, we’ll let Anaf explain:

Be sure to visit the uTest Youtube Channel to see the rest of our QUEST 2012 interviews, including time with Ann Hungate, Director of Quality for Nationwide Insurance, and Microsoft Evangelista Angela Dugan.