Why Security Testing Is So Important

Voting BoothsYou can do just about anything online these, so much so that it feels like an inconvenience if you can’t complete a task online. But some things are just best left the old fashioned way.

Take, for example, the act of voting. I’m not talking about voting for American Idol (which you actually can do online now), I’m talking about voting in a major, official election. While paper absentee ballots may seem outdated, voting has proved to fragile and tamper-tempting to be shifted online. We wouldn’t know that though without some good, solid security testing.

A few years ago an e-voting system was created for Washington, D.C. and in 2010 its developers reached out to security testing experts to put the system through its paces. It failed miserably. The story is surfacing again now because the processes and results of the testing were recently officially published. The testers didn’t find some exceptionally complicated flaw only detectable with a lot of out-of-the-box thinking, they were able to completely infiltrate and manipulate the program. There’s The H with some details:

“Within 48 hours of the system going live, we had gained near complete control of the election server”, the researchers wrote in a paper that has now been released. “We successfully changed every vote and revealed almost every secret ballot.” The hack was only discovered after about two business days – and most likely only because the intruders left a visible trail on purpose. …

The security experts investigated common vulnerable points such as login fields, the virtual ballots’ content and file names, and session cookies – and found several exploitable weaknesses. Even the Linux kernel used in the project proved to have a well known vulnerability. They were also able to use the PDFs generated by the system to trick the encryption mechanism, while unsecured surveillance cameras provided additional insights into the infrastructure. While the open source nature of the code made their work somewhat easier, they believe that attackers would have been able to make quick headway even if the system had been proprietary.

Continue Reading

7 Tips for Stellar Test Management

Tips to excel at test managementWe all know what happens when you rush through testing and push a new product out to market too early (hint: crashing and burning comes to mind). In the spirit of not releasing software with major security flaws, functional defects or usability missteps Traq Software has highlighted “7 Important Principals for Test Management.” Full disclosure: Traq Software sells QA management software, but the tips are good too keep in mind non-the-less. (Numbers 4 and 5 are my favorites!)

1. Make sure you have a repeatable process. A good process helps you see where you are and where you are going.

2. Don’t cut corners prior to a release. When the delivery date is getting closer there is, naturally, a tendency to want to skip some low priority test management tasks. In doing so you hope to get the product out on time. Resist this temptation.

3. Know the metrics. Defect find rates, cases executed and lines of code changed. All these metrics help you argue the case for why the product may, or may not, be ready for release.

4. Listen to the testers. Software testers are your projects headlights. They light the road at night and help you read the map. They are trying to help you get to your destination in the shortest amount of time. Ignore their advice and you can be sure you’ll end up taking the longest route to your destination.

5. Employ a good QA manager. The QA manager is like the pilot of the plane guiding the product to touch down. … good QA manager has an eye for balancing the demands of time, quality and features. He or she is worth listening too carefully.

6. Get the customer involved. The sooner your customer starts giving you feedback the sooner you can correct issues.

Continue Reading

Announcing the 2011 uTester of the Year Awards

2011 uTester of the Year AwardsToday, we’re thrilled to announce the results of our third annual uTester of the Year Awards. Every year, we recognize uTesters who have consistently gone above and beyond their call of duty in their participation with uTest projects. This year’s winners were selected by our community and project management teams, who have had the privilege of working closely with such an extremely talented community of professional testers. From test automation to test team lead, these winners are truly experts in a variety of testing domains. The level of talent continues to impress, with each year’s accolades becoming more and more difficult to attain (and judge). So without further ado, let’s meet our 2011 winners!

Top honor for the 2011 award goes to David Honeyball from the United Kingdom!

David joined uTest in June of 2009. Since joining us, David has become a Gold rated tester in Functional, Load and Localization testing, as well as having achieved the silver rating in both Usability and Security testing. David also became our Top Test Team Lead in 2011, successfully leading nearly 200 projects alongside of uTest project managers. David had this to say about his experience with uTest during the past year:

I joined uTest back in the summer of 2009 and have to say I have never looked back. What started out as something extra in my spare time has taken up more and more of my time in a good way and has increased my confidence as a tester to levels I never thought possible. I have been a tester for nearly 15 years but can safely say that every day with uTest is a new experience and a new challenge.

I have met so many wonderful people including other testers, CMs, PMs and customers who are all committed to achieving their goals and creating a wonderful service. They have guided me and helped me in the last year and increased my communication skills as well.

TTL (Test Team Lead) Experience:

2011 has been a big success for me personally, as it has for uTest in regards to growth and development, in many ways due to my journey from tester to TTL. I started out as a TTL early in the year and felt at the time that it was a great way forward and would be of huge assistance and help to customers. Since then I have been involved in something nearing 100 cycles as TTL, but that could be more now!  I enjoy the TTL role immensely as I get to speak to the PMs and testers and help out others who are stuck. I hope if you have worked with me that you know that I take that side of things very seriously. One of the great rewards is helping someone who is stuck to complete a test case or test for example. Above all, I believe that with my experience I am a very fair TTL and have the best interests of customer and testers close at heart.

As time goes on I hope to grow more and more into this role and improve further as there is still so much to learn.

Special Projects 

Apart from testing and TTL work which does take up some time as you can imagine, I also help with test case writing for certain PMs which I enjoy and get value from. I was also heavily involved in the startup of the test case conversion to uTest which is the new system we see today. This side of things linked in with my testing and other roles forms a vital component of what I do as well and hope it adds value to the company as a whole.

So, just want to say a huge thanks to everyone involved with uTest for the opportunity and long may we continue to grow together and become invaluable to customers across the globe…

The complete list of winners is shown below:

Continue Reading

Software Engineering Hits High School

Software Engineering Hits High SchoolA teacher in Massachusetts dedicated a computer class to developing and testing mobile apps. The Education Secretary in the UK is calling for a total program overhaul of country’s computer education curriculum. Now, the Mayor Michael Bloomberg of New York City has declared that an entire public high school will be devoted to teaching students software engineering. From Government Computer News:

“Today, far too many of our graduates are leaving without the skills they need to succeed beyond high school. Not every student wants to go to college, nor is college right for everyone. But all students should leave prepared to succeed in the next phase of their lives,” Bloomberg said. “It’s a new way of thinking about secondary school based on today’s economic realities.” …

Frank Thomas, a spokesman for the city’s Department of Education, anticipates that the school will have between 420 and 460 students by 2015, when all four grade levels are enrolled, Adrianne Jeffries reported in BetaBeat. The school will start with a ninth-grade class this year and add on another grade level for the next three years.

The city has other specialized high schools for science, math, the performing arts and other subjects, but it did not have one focused on computer science. …

Joel Spolsky, a board member of the new school, said one reason he’s a proponent of the school is that it could can train many excellent software engineers who are not currently at the top of their class academically.

“I think this is the best thing about the school,” he said in a blog post. “A lot of kids are just not interested enough in other academic subjects to get good grades, but they would make great software engineers. A lot of immigrants (especially in New York) are not yet proficient enough in English to get good grades in all their subjects, but they’re going to make great software engineers, too.”

I have to say, one instance is cool. Two instances make you raise an eyebrow. Three instances (especially when they’re consistently bigger examples) might just be the start of a trend. And this trend of focusing not only on computer basics, but on more advanced – more engaging – computer topics that can lead to lucrative, fulfilling career paths is long over due.

Celebrating a major milestone in our Software Testing Community

50,000+ TestersWhile our usual maniacal focus is on quality over quantity, it’s not unreasonable to recognize a major milestone that occurred today, January 18, 2012: surpassing 50,000 testers in the uTest community! Just to be clear, that’s over 50,000 testers from 185 countries around the world – from experts in automation to gurus in usability testing. Here are several other facts about our community:

  • Every month, there are approximately 1,000 new tester registrations
  • Over 99.9% of these registrations are organic – word of mouth, tradeshows and conferences, tester referrals
  • The majority of testers span rather evenly across North America, Europe, and Asia. The rest fill out in South America, Africa, and Australia
  • Over 80% of uTesters have a Bachelor’s degree or higher
  • uTesters bring a wealth of knowledge and diverse set of skills to the table: creating test cases, usability surveys, load and performance scripts, automation scripts, security coverage reports, usability audits and expert reviews; executing test plans, usability surveys, live load test cases, security scans, exploratory tests, and translation tasks and proofs

And…back to our maniacal attention to quality. Although there is certainly strength in numbers and meaning to this milestone, the real excitement stems from the various “homegrown” programs that shape our crowdsourcing model. Less than a year ago, we announced several new initiatives that have transformed the uTest community from an unruly crowd to one that is self-sufficient, self-teaching and self-policing. From paid leadership roles for our top testers to unpaid auditions for newbie testers, there is a role for nearly everyone and a path for the most ambitious. And now that most of us have embraced the New Year, it’s only fitting that there are new programs just around the corner – ones that leverage the foundation built in the past year and continue to benefit our community at large. More details to come shortly!

For now, please join me in raising your glass to celebrate this major milestone with us!

Flypaper for Software Bugs

Pinterest Facebook Production Bug Screen ShotOne of the biggest fears of companies developing new software or app or launching a new website is that some fundamental bug will slip through the testing cracks and only rear its ugly head post-launch.

That fear is compounded these days now that review sites and social media make it effortless for dissatisfied customers to voice their grievances not only to their friends, co-workers and next store neighbors, but to all the friends, co-workers, next store neighbors they’ve ever known. Plus a slue of strangers they don’t actually know.

Now, in addition to written complaints and bad reviews, the general public can share images of software bugs. Check out this story on TechCrunch about a guy who posted images of Facebook bugs on Pinterest (think of his board as flypaper for Facebook bugs):

Former Facebook engineer (and current Phabricator creator) Evan Priestley has taken the opposite route; In the spirit of coding excellence, Priestly has created a Pinterest log of over 30 Facebook bugs he’s tracked since September 2011. It’s really impressive.

Continue Reading

Where’s the Cinnabon?… or, Will Indoor LBS Hit it Big in 2012?

‘Tis the season to prognosticate.

We’re 17 days away from the new year, and far before Auld Lang Syne begins playing and we pretend to know the words (after all the champagne, who can remember the lyrics we optimistically Google’d the day before anyways?), we’re pondering what changes are in store for us the next twelve months.

In a whitepaper released by ABI Research this week, their tech analysts took a collective look into the crystal ball for 2012 and (in their words) “have drawn some bold lines in the sand on a plethora of top-of-mind topics.”

But instead of predicting what WOULD happen in the mobile and telecom space, they took a different spin on the usual list and forecasted what WOULDN’T happen.  Nice twist.  (And a really good read.)

One of their more interesting predictions for those of us in software testing is by Patrick Connolly, Senior Analyst of Telematics and Navigation:  “Indoor location will NOT become commonplace in 2012.” 

It’s easy to see how this could be true…but also surprising.

After all, for as many articles that have been written about the technological challenges in making Indoor Location Based Services (LBS) a reality, there has been an equal amount of big name, big buzz announcements about it over the past few months.  There are dozens of industry-leading companies—including Apple, Navteq, Qualcomm and Nokia—tackling the challenge from every angle.

There are even some major apps launching to give Indoor LBS a jolt from vision to reality.  For instance, Google announced on their Mobile blog in November that the new Google Maps 6.0 gives users (on Android OS 2.1 mobile devices) the ability to Map the Vast Indoors, vis-à-vis:

Continue Reading

Guest Post: How Acquia Tests Software (via uTest)

In case you missed today’s news, uTest announced an exciting new partnership with Acquia, the enterprise guide to Drupal. As part of the deal – which provides their customers with unique access to uTest’s full suite of testing services – Acquia was legally obligated to write a guest post for our blog. Actually, that wasn’t part of the deal, but we were able to get a great guest post from them anyway.

Meet Stellina McKinney – Acquia’s Director of Engineering Services – who is here to discuss how Acquia leverages the uTest community. That’s right, not only is Acquia a uTest partner, they’re also a very active uTest customer. Enjoy the post!

*******

I started at Acquia 6 months ago, having previously worked for larger, process-heavy corporations that sold packaged proprietary, software with long release cycles. Our QA teams consisted of over 50 people (sometimes a lot more), and were always the long pole in the process, whether it was Agile or Waterfall.

Not so at Acquia.

At Acquia, I manage a lean QA team of 4 people (we have another team that tests usability), and we support 5 products. We work in an Agile environment, release every 3 weeks, and meet our quality goals for each sprint.

Our QA testing strategy at Acquia is to perform tests on agile user stories (akin to use-case tests or acceptance tests in Behavior Driven Development). Our goals are to:

  • Define the behavior of the system, and not have a previously-coded system define the behavior for us
  • Test failure cases so that they won’t affect production
  • Stress systems through performance and load automation
  • Mix automated and manual testing methods, as they’re complementary (machines are fast and consistent, but people have brains and are unpredictable)

We do this by:

  • Listing the scenarios that must succeed for a product to be complete
  • Writing automated tests to perform basic success and failure operations
  • Engaging a crowd-sourced manual testing platform to examine our product in more depth

How can we do this with only a team of 4?  uTest’s crowdsourced testing platform lets us leverage over 45 testers a month, without exceeding my start-up budget.

Continue Reading

The Relationship Between Testers and Programmers

Testers and programmers are two groups of people who should get along, but often don’t. It’s a sad fact of life that testers (by virtue of what they do) often bring bad news. And programmers, by virtue of what they do, are the source of the defects that create the bad news. Rather than both accepting that this is a reality of life and working together, they allow the relationship to become acrimonious.

James Bach is no stranger to this problem, and his latest blog post is a blueprint for making that relationship more productive and professional. Titled A Tester’s Commitments, James starts by writing:

Dear Programmer,

My job is to help you look good. My job is to support you as you create quality; to ease that burden instead of adding to it.

What follows are twelve commitments a tester should make to their programmers. They include things like:

  • I provide a service. You are an important client of that service. I am not satisfied unless you are satisfied.
  • I will learn the product quickly, and make use of that knowledge to test more cleverly.
  • I will not carelessly waste your time. Or if I do, I will learn from that mistake.

But James is not in usual form unless he invites controversy, and that first bullet struck quite a chord with some of his readers. Testers provide a service!? Since when?

Continue Reading

Holiday Shopping on Mobile. Even the Elves Need an App for That.

Ho, ho, ho!  Whoa there, Blitzen– wasn’t it just Halloween?  It sure feels that way. After all, I still have two pounds of trick-or-treat candy to pretend I’m not eating.

Unfortunately, my four-year-old has already implored me to take down the skeleton and spiders hanging in the doorway because they’re going to scare away Santa.  So, rather than arguing the salient fact that Santa shimmies down the chimney versus ringing the doorbell, I’ve officially started morphing decor from the marvelous macabre to merry old Saint Nick.  Kids: 1. Mom: 0.

Nonetheless, the fact hasn’t escaped me that we’re two weeks away from Cyber Monday (November 28th), an occasion that online retailers have been planning for months.  Since summer, global brands and independent e-tailers have been testing and re-testing their mobile apps and web sites for functionality, usability, localization glitches and possible bottlenecks in site performance that could jeopardize their revenue potential.

Moreover, the ante has been upped now that the iPad and other tablets have entered the scene.  Online retailers that spent the last few years optimizing their mobile apps and porting them to additional platforms like Android, are now going through the process from scratch with tablets.  Not only are the specs non- standardized, varying significantly by manufacturer, device and network performance like smartphones.

Continue Reading