Software Testing Blog

It’s that time of year again – time for family, friends and food. Yes, the holiday season is upon us, and around the halls of uTest, we’re feeling particularly jolly.  So how does a company with 49,000 testers from 183 countries around the globe say “happy holidays”?  Well here at uTest, this is how we roll:

And now, for a little holiday fun, here’s our naughty & nice list from around the 2011 apps universe:

Naughty List (tsk, tsk, tsk):

• India’s government, who’s recent software glitch led them to overstate exports by $9,000,000,000. Personally, we think that bugs should never create problems that exceed $8.5 billion, but that’s just us. And earlier this year, the Indian government wrestled with PayPal over payments to the country’s online merchants. Santa never forgets… I mean, he makes a list and checks it twice (which is his form of QA).
• The U.S. politicians driving SOPA. Yes, the same tech visionaries who brought us such timeless quotes as “the internets” and “the series of tubes” have decided to try their hand at defeating online piracy with a proposed bill that many believe would cripple innovation and free speech on the web.
• Pretty much everything on this list of “worst software glitches of 2011” from WebLayers and ZDNet. And this list from Appolicious. And this one from AppAdvice.
• TripAdvisor founder, Stephen Kaufer, who chatted with TechCrunch about “404 testing”, in which they add a link for a feature that doesn’t exist (and returns the dreaded 404 message) to determine user demand for features in a web app. Uh, guys, congrats on the IPO, but there’s a much better way to quickly build and test new features without subjecting users to an intentionally lousy experience.
• The Australian casino that refused to pay a customer the $57MM he “won”, because they blamed his “winning” on a bug in the slot machine software.

Nice List (or at least people we want to give a hearty holiday thanks to):

• Our testers: the aforementioned 49,000 testers from 183 countries around the world. Because of you, we’re enabling nearly 1,000 companies to launch apps that work as well in the wild as they do in the QA lab. Big ups.
• Our customers: it’s nice to know that there are companies and tech execs that care so much about their end user experience that they’re willing to pay us to point out every glitch, bug and snafu we can find – before their app gets into the hands of users. You rock.
• Our investors: we’ve always loved you, but the recent $17MM D round was the gift that keeps on giving — and that enables us to grow faster, grow bigger, change the world of apps, and all that other stuff we promised we’d do.
• Our blog readers: we’re psyched that there are about 4X as many of you as there were at this time last year (keep telling your friends, fans and followers). And since you’ve read this far down in the post, it would feel awkward not to include you amongst our list of favorite people!
• Ben Huh, Paul Graham, the crews at Scribd and Tumblr, and  for taking the fight against SOPA to the streets. Way to stand up for a cause you believe in, and to do it in bold, engaging ways. Santa likes that.

Ok, all kidding aside, uTest sincerely wishes everyone a peaceful, safe and happy holiday season. It’s been an amazing 358 days thus far in 2011, and we’re supremely thankful that we get to spend our days and nights at the epicenter of this crazy (but never dull) apps universe with all of you.

Earlier in December Rafe Needleman, a writer at CNet, began posting daily gems of advice called “Startup Secrets.” They are short and sweet and include a memorable quote from a variety of people. So far, we’re up to 12 secrets. While they’re all good, I wanted to highlight a few particularly pertinent to software and mobile development and testing.

Startup Secret No. 3: Support is Product

“Make sure the support fits the product.”
–Josh Weinberg, The Digital Life Consulting Group

So, today’s puzzlers are: Who are your users? Where are they? How do they use your product? And how can you make the experience of using your support as close as possible to using the product itself?

Startup Secret No. 6: Live the Shirt

“Believe in your f***ing self” –Dave McClure‘s T-Shirt

I love Dave’s shirt, because this is what entrepreneurship is really about. Pushing, pushing, pushing. Pushing your idea, pushing yourself, and pushing through the dismissive and small-minded negativity from people who don’t share your passion.

Read more…

We’re excited to share the details of our 5th uMeetup event – held in Singapore. In case you missed the details of our other 4 uMeetup events, you check them out here. uMeetups are local networking events organized by top testers and sponsored by uTest. Invitations are extended to all software testers in the local vicinity to meet in person, listen to presentations on special software testing topics, and learn more about participating in paid projects with uTest.

Bryan Loh, a gold uTester organized this latest uMeetup. Despite it raining heavily, 9 uTesters were able to meet up at a local coffee shop not far from the famous “Fullerton Hotel”. uTesters in attendance discussed the history of uTest, personal experiences on test cycles, tactics they’ve employed for getting more projects, MVT awards, and what they thought was the best resource available to uTesters: the forums.

We’ve received some great feedback from testers who have taken part in uMeetups in the past. Being passionate about testing and about uTest seems to be a common thread that brings these testers together.

Want to have a uMeetup in your area? Check the forums to find the one nearest to you. If you don’t see your country on the list  apply for a sponsorship here.

Take a look at a  few pictures from the event after the jump:

Read more…

For the twelfth bug of Christmas, my manager said to me:

1. Tell them it’s a feature
2. Say it’s not supported
3. Change the documentation
4. Blame it on the hardware
5. Find a way around it
6. Say they need an upgrade
7. Reinstall the software
8. Ask for a dump
9. Run with the debugger
10. Try to reproduce it
11. Ask them how they did it and
12. See if they can do it again!

- By Alam Saeed

For more testing humor, check out Computerjokes.net

IBM released this year’s “5 in 5” list – a tradition where they make five, slightly science fiction-y, predictions about tech we’ll have in the next five years. This year’s list:

1. People power will come to life
2. You will never need a password again
3. Mind reading is no longer science fiction
4. The digital divide will cease to exist
5. Junk mail will become priority mail

While those seem awesome, some of them also seem pretty far fetched (mind reading?). But as it turns out … most of these concepts are not only in the works, they’re already here (albeit in a rough form). Let’s take a look:

1. People power will come to life.

IBM says:
“In five years you be able to power your house with the energy you create yourself.”

How it’s already here:
PaveGen tiles have been turning the kinetic energy of footsteps into harvest-able electricity for a few years now and 20 of the tiles will be in place at a busy pedestrian mall in London in time for the 2012 Olympics. The slabs are expected to generate enough power to light half of the location’s outdoor lamps.

Read more…

You thought IE6 would die after Facebook, Google and hundreds of other companies stopped supporting it. You thought IE6 was dead when they held a mock funeral for it in 2010. You thought it would be even more dead (deader?) when just last week, we learned that Microsoft will be automatically upgrading users to the latest version of the web browser. You were wrong. To quote Richard Pryor: I Ain’t Dead Yet, Mother#*%$@!!

And neither is IE6. In the past, we’ve called IE6 the zombie browser that can’t be killed and it’s certainly lived up to that moniker. However, regarding that last piece of news (Microsoft’s upcoming automatic updates) it looks like IE6 may finally get phased out. For real this time. Or maybe not.

Here’s Webmonkey.com with the details:

The new auto-update feature will only apply to users who’ve opted into the automatic updates through Windows Update. Those that have opted in will be upgraded to the latest version of IE available for their system. If you’re still on Windows XP that means you’ll be updated to IE 8. Vista and Windows 7 users will move to IE 9. The Windows Blog notes that when upgrading, your home page, search provider, and default browser settings will not be affected.

Web developers still supporting IE 6 may not need to do so much longer if Microsoft’s auto-update strategy pays off. Since the new auto-update mechanism will apply to IE 7 as well, it too may not need to be supported much longer. Of course, even in the best case scenario where IE 6 and 7 users drop below 5 percent worldwide, web developers would still need to contend with IE 8. While IE 8 was a huge step up from its predecessors, it still lacks support for most of the HTML5 and CSS 3 features found in modern web browsers.

Somehow, IE6 usage currently stands at 8.4% worldwide. That includes a whopping 27.9% in China. As such, we’re going to hold off on pronouncing IE6 dead for the time being. However, the browser’s days are clearly numbered. Right?

In part II of our Testing the Limits interview with security expert Richard Stiennon, we get his thoughts on where companies are least prepared for a security breach; the explosion of mobile related threats; the future of cyber security and much more. If you missed the first installment, read Part I now.

uTest: What’s the one piece of security advice you would have for companies that are in their infancy?

RS: Plan ahead. As you grow so does your “target area.” When you launch your product or service you may not be in the cross hairs of a bad actor. But you will be. Sometimes security builds in “friction” that can slow down customer acquisition. So you may not want to require 12 character passwords and SMS authentication at the beginning but know that eventually you will have to deal with account theft, breaches, and spammers. Have a plan so you can implement defenses quickly; and ideally before a damaging attack or breach. You don’t want to be in the position of the Sony Play Station Network that implemented protections *after* a series of attacks that cost $170 million to recover form.

uTest: You’ve said before that mobile will not require its own anti-virus systems. That said, it seems that mobile threats are multiplying by the hour. In your view, what’s the biggest security challenge in terms of mobile?

RS: Apps, apps, apps. VPNs, firewalls, and carrier filtering are going to impede network based attacks. Containing and vetting applications is the biggest security challenge for the platform vendors.

uTest: Looking back to the 1990s, what’s surprised you the most about the evolution of cyber security? What’s been your biggest disappointment?

RS: For me the biggest surprise was the confluence of existing criminal organizations and cyber crime, especially arising out of the demise of the Soviet Union. In retrospect is seems obvious, but at the time it was a wake-up call for me that guys with guns and baseball bats were going high-tech. My biggest disappointment is that security has never become important enough to spawn secure networks or secure computers. Not a single ISP or carrier has gone to market with a secure network with complete content inspection. Not a single computer manufacturer outside the military has sold a computer whose primary feature is security. They are all happy to sell you security add-ons but not willing to step up and address the underlying vulnerability of their products.

uTest: What happens in the next decade of security is anyone’s guess, but your predictions carry a bit more weight. Care to make any bold predictions on the future of cyber security? The bolder, the better.

Read more…

What an honor it is to have security expert Richard Stiennon cap off another great year of Testing the Limits. Aside from being the most followed IT security analyst on Twitter, Richard is an accomplished writer, having authored Surviving Cyberwar and the soon to be published Cyber Defense: Countering Targeted Attacks. Richard is currently Chief Research Analyst of IT-Harvest, a security analyst firm with a wide range of high-profile clients. For more on his background, click here.

In part I of our interview, we get his thoughts on the difference between security and other types of testing; what the world would look like under full blown cyber war; the biggest threats to the typical web user; the motives of hackers and more. Tune in tomorrow for part II.

uTest: You started out in the field of aerospace as an engineer and wound up as one of the world’s top security experts (so typical). Kidding aside, what attracted you to the field of cyber security? And what’s kept you there for the better part of two decades?

Richard Stiennon: My transition from structural engineer to networking came when I started a dial-up ISP in Michigan, but I did not get the security bug until joining Netrex, which was an integrator of security products and services. Through Netrex I worked with a lot of the early security products and the founders of ISS, and Check Point Software. By the time Netrex was acquired by ISS (later to be acquired by IBM) I had moved on to PwC where I got exposed to large enterprise and performed audits on their security postures. From there it was on to Gartner and after that I was firmly entrenched in the IT security world. I have a low threshold for boredom. The security industry moves so fast you get left behind if you allow your eyes to glaze over for a second.

uTest: Is it fair to say that security testing requires a much different mindset/persona than other types of testing? If so, what specific qualities and characteristics are needed in a security tester? If you were assembling a team of security testers, what traits would you look for?

RS: Security testing of software throughout its development cycle is indeed different than quality and functionality testing. Instead of testing against end user use cases you have to have a mind set of an attacker, a completely different use case. In addition to meticulous use of security testing tools (HP-Fortify, Veracode, etc) a security tester must understand the application and how an attacker would leverage built-in functionality to subvert a system. A security tester must be diligent and detail oriented as well as imaginative and wily – a rare combination.

uTest: In 2010, you published Surviving Cyber War, which gave the world an inside look into the onset of state-sponsored cyber war. Since then, there’s been no shortage of similar incidents (Stuxnet, Anonymous, to name a few). Are you surprised at the speed in which cyber warfare is evolving?

Read more…

‘Tis the season to prognosticate.

We’re 17 days away from the new year, and far before Auld Lang Syne begins playing and we pretend to know the words (after all the champagne, who can remember the lyrics we optimistically Google’d the day before anyways?), we’re pondering what changes are in store for us the next twelve months.

In a whitepaper released by ABI Research this week, their tech analysts took a collective look into the crystal ball for 2012 and (in their words) “have drawn some bold lines in the sand on a plethora of top-of-mind topics.”

But instead of predicting what WOULD happen in the mobile and telecom space, they took a different spin on the usual list and forecasted what WOULDN’T happen.  Nice twist.  (And a really good read.)

One of their more interesting predictions for those of us in software testing is by Patrick Connolly, Senior Analyst of Telematics and Navigation:  “Indoor location will NOT become commonplace in 2012.” 

It’s easy to see how this could be true…but also surprising.

After all, for as many articles that have been written about the technological challenges in making Indoor Location Based Services (LBS) a reality, there has been an equal amount of big name, big buzz announcements about it over the past few months.  There are dozens of industry-leading companies—including Apple, Navteq, Qualcomm and Nokia—tackling the challenge from every angle.

There are even some major apps launching to give Indoor LBS a jolt from vision to reality.  For instance, Google announced on their Mobile blog in November that the new Google Maps 6.0 gives users (on Android OS 2.1 mobile devices) the ability to Map the Vast Indoors, vis-à-vis:

Read more…

Last month there were several reports of cyber attacks on water treatment plants ( Houston, TX and Springfield, IL come immediately to mind). The Springfield incident turned out to be a major miscommunication, but the Houston attack is holding strong and at least three other attacks have been confirmed by the FBI. These attacks were so real, in fact, that Michael Welch, deputy director of the FBI’s Cyber Division, recently announced that the FBI will be increasing its cyber budget by roughly 12%. Here’s a recap from Sophos’ Naked Security blog:

At a recent security conference Michael Welch, the deputy assistant director of the FBI’s Cyber Division, gave a speech where he discussed the issue of SCADA security.

Information Age magazine reported on his speech and quoted Welch as saying:

"We just had a circumstance where we had three cities, one of them a major city within the US, where you had several hackers that had made their way into SCADA systems within the city."

… It’s great that Welch acknowledges the work we have to do in this area and even went so far as to suggest the FBI will double the size of their Cyber division in the next 12 to 18 months.

Sound too good to be true? Then it probably is.

Read more…