Software Testing Blog

Today, we’re thrilled to announce the results of our third annual uTester of the Year Awards. Every year, we recognize uTesters who have consistently gone above and beyond their call of duty in their participation with uTest projects. This year’s winners were selected by our community and project management teams, who have had the privilege of working closely with such an extremely talented community of professional testers. From test automation to test team lead, these winners are truly experts in a variety of testing domains. The level of talent continues to impress, with each year’s accolades becoming more and more difficult to attain (and judge). So without further ado, let’s meet our 2011 winners!

Top honor for the 2011 award goes to David Honeyball from the United Kingdom!

David joined uTest in June of 2009. Since joining us, David has become a Gold rated tester in Functional, Load and Localization testing, as well as having achieved the silver rating in both Usability and Security testing. David also became our Top Test Team Lead in 2011, successfully leading nearly 200 projects alongside of uTest project managers. David had this to say about his experience with uTest during the past year:

I joined uTest back in the summer of 2009 and have to say I have never looked back. What started out as something extra in my spare time has taken up more and more of my time in a good way and has increased my confidence as a tester to levels I never thought possible. I have been a tester for nearly 15 years but can safely say that every day with uTest is a new experience and a new challenge.

I have met so many wonderful people including other testers, CMs, PMs and customers who are all committed to achieving their goals and creating a wonderful service. They have guided me and helped me in the last year and increased my communication skills as well.

TTL (Test Team Lead) Experience:

2011 has been a big success for me personally, as it has for uTest in regards to growth and development, in many ways due to my journey from tester to TTL. I started out as a TTL early in the year and felt at the time that it was a great way forward and would be of huge assistance and help to customers. Since then I have been involved in something nearing 100 cycles as TTL, but that could be more now!  I enjoy the TTL role immensely as I get to speak to the PMs and testers and help out others who are stuck. I hope if you have worked with me that you know that I take that side of things very seriously. One of the great rewards is helping someone who is stuck to complete a test case or test for example. Above all, I believe that with my experience I am a very fair TTL and have the best interests of customer and testers close at heart.

As time goes on I hope to grow more and more into this role and improve further as there is still so much to learn.

Special Projects 

Apart from testing and TTL work which does take up some time as you can imagine, I also help with test case writing for certain PMs which I enjoy and get value from. I was also heavily involved in the startup of the test case conversion to uTest which is the new system we see today. This side of things linked in with my testing and other roles forms a vital component of what I do as well and hope it adds value to the company as a whole.

So, just want to say a huge thanks to everyone involved with uTest for the opportunity and long may we continue to grow together and become invaluable to customers across the globe…

The complete list of winners is shown below:

Read more…

A teacher in Massachusetts dedicated a computer class to developing and testing mobile apps. The Education Secretary in the UK is calling for a total program overhaul of country’s computer education curriculum. Now, the Mayor Michael Bloomberg of New York City has declared that an entire public high school will be devoted to teaching students software engineering. From Government Computer News:

“Today, far too many of our graduates are leaving without the skills they need to succeed beyond high school. Not every student wants to go to college, nor is college right for everyone. But all students should leave prepared to succeed in the next phase of their lives,” Bloomberg said. “It’s a new way of thinking about secondary school based on today’s economic realities.” …

Frank Thomas, a spokesman for the city’s Department of Education, anticipates that the school will have between 420 and 460 students by 2015, when all four grade levels are enrolled, Adrianne Jeffries reported in BetaBeat. The school will start with a ninth-grade class this year and add on another grade level for the next three years.

The city has other specialized high schools for science, math, the performing arts and other subjects, but it did not have one focused on computer science. …

Joel Spolsky, a board member of the new school, said one reason he’s a proponent of the school is that it could can train many excellent software engineers who are not currently at the top of their class academically.

“I think this is the best thing about the school,” he said in a blog post. “A lot of kids are just not interested enough in other academic subjects to get good grades, but they would make great software engineers. A lot of immigrants (especially in New York) are not yet proficient enough in English to get good grades in all their subjects, but they’re going to make great software engineers, too.”

I have to say, one instance is cool. Two instances make you raise an eyebrow. Three instances (especially when they’re consistently bigger examples) might just be the start of a trend. And this trend of focusing not only on computer basics, but on more advanced – more engaging – computer topics that can lead to lucrative, fulfilling career paths is long over due.

While our usual maniacal focus is on quality over quantity, it’s not unreasonable to recognize a major milestone that occurred today, January 18, 2012: surpassing 50,000 testers in the uTest community! Just to be clear, that’s over 50,000 testers from 185 countries around the world – from experts in automation to gurus in usability testing. Here are several other facts about our community:

• Every month, there are approximately 1,000 new tester registrations
• Over 99.9% of these registrations are organic – word of mouth, tradeshows and conferences, tester referrals
• The majority of testers span rather evenly across North America, Europe, and Asia. The rest fill out in South America, Africa, and Australia
• Over 80% of uTesters have a Bachelor’s degree or higher
• uTesters bring a wealth of knowledge and diverse set of skills to the table: creating test cases, usability surveys, load and performance scripts, automation scripts, security coverage reports, usability audits and expert reviews; executing test plans, usability surveys, live load test cases, security scans, exploratory tests, and translation tasks and proofs

And…back to our maniacal attention to quality. Although there is certainly strength in numbers and meaning to this milestone, the real excitement stems from the various “homegrown” programs that shape our crowdsourcing model. Less than a year ago, we announced several new initiatives that have transformed the uTest community from an unruly crowd to one that is self-sufficient, self-teaching and self-policing. From paid leadership roles for our top testers to unpaid auditions for newbie testers, there is a role for nearly everyone and a path for the most ambitious. And now that most of us have embraced the New Year, it’s only fitting that there are new programs just around the corner – ones that leverage the foundation built in the past year and continue to benefit our community at large. More details to come shortly!

For now, please join me in raising your glass to celebrate this major milestone with us!

One of the biggest fears of companies developing new software or app or launching a new website is that some fundamental bug will slip through the testing cracks and only rear its ugly head post-launch.

That fear is compounded these days now that review sites and social media make it effortless for dissatisfied customers to voice their grievances not only to their friends, co-workers and next store neighbors, but to all the friends, co-workers, next store neighbors they’ve ever known. Plus a slue of strangers they don’t actually know.

Now, in addition to written complaints and bad reviews, the general public can share images of software bugs. Check out this story on TechCrunch about a guy who posted images of Facebook bugs on Pinterest (think of his board as flypaper for Facebook bugs):

Former Facebook engineer (and current Phabricator creator) Evan Priestley has taken the opposite route; In the spirit of coding excellence, Priestly has created a Pinterest log of over 30 Facebook bugs he’s tracked since September 2011. It’s really impressive.

Read more…

‘Tis the season to prognosticate.

We’re 17 days away from the new year, and far before Auld Lang Syne begins playing and we pretend to know the words (after all the champagne, who can remember the lyrics we optimistically Google’d the day before anyways?), we’re pondering what changes are in store for us the next twelve months.

In a whitepaper released by ABI Research this week, their tech analysts took a collective look into the crystal ball for 2012 and (in their words) “have drawn some bold lines in the sand on a plethora of top-of-mind topics.”

But instead of predicting what WOULD happen in the mobile and telecom space, they took a different spin on the usual list and forecasted what WOULDN’T happen.  Nice twist.  (And a really good read.)

One of their more interesting predictions for those of us in software testing is by Patrick Connolly, Senior Analyst of Telematics and Navigation:  “Indoor location will NOT become commonplace in 2012.” 

It’s easy to see how this could be true…but also surprising.

After all, for as many articles that have been written about the technological challenges in making Indoor Location Based Services (LBS) a reality, there has been an equal amount of big name, big buzz announcements about it over the past few months.  There are dozens of industry-leading companies—including Apple, Navteq, Qualcomm and Nokia—tackling the challenge from every angle.

There are even some major apps launching to give Indoor LBS a jolt from vision to reality.  For instance, Google announced on their Mobile blog in November that the new Google Maps 6.0 gives users (on Android OS 2.1 mobile devices) the ability to Map the Vast Indoors, vis-à-vis:

Read more…

In case you missed today’s news, uTest announced an exciting new partnership with Acquia, the enterprise guide to Drupal. As part of the deal – which provides their customers with unique access to uTest’s full suite of testing services – Acquia was legally obligated to write a guest post for our blog. Actually, that wasn’t part of the deal, but we were able to get a great guest post from them anyway.

Meet Stellina McKinney – Acquia’s Director of Engineering Services – who is here to discuss how Acquia leverages the uTest community. That’s right, not only is Acquia a uTest partner, they’re also a very active uTest customer. Enjoy the post!

*******

I started at Acquia 6 months ago, having previously worked for larger, process-heavy corporations that sold packaged proprietary, software with long release cycles. Our QA teams consisted of over 50 people (sometimes a lot more), and were always the long pole in the process, whether it was Agile or Waterfall.

Not so at Acquia.

At Acquia, I manage a lean QA team of 4 people (we have another team that tests usability), and we support 5 products. We work in an Agile environment, release every 3 weeks, and meet our quality goals for each sprint.

Our QA testing strategy at Acquia is to perform tests on agile user stories (akin to use-case tests or acceptance tests in Behavior Driven Development). Our goals are to:

• Define the behavior of the system, and not have a previously-coded system define the behavior for us
• Test failure cases so that they won’t affect production
• Stress systems through performance and load automation
• Mix automated and manual testing methods, as they’re complementary (machines are fast and consistent, but people have brains and are unpredictable)

We do this by:

• Listing the scenarios that must succeed for a product to be complete
• Writing automated tests to perform basic success and failure operations
• Engaging a crowd-sourced manual testing platform to examine our product in more depth

How can we do this with only a team of 4?  uTest’s crowdsourced testing platform lets us leverage over 45 testers a month, without exceeding my start-up budget.

Read more…

Testers and programmers are two groups of people who should get along, but often don’t. It’s a sad fact of life that testers (by virtue of what they do) often bring bad news. And programmers, by virtue of what they do, are the source of the defects that create the bad news. Rather than both accepting that this is a reality of life and working together, they allow the relationship to become acrimonious.

James Bach is no stranger to this problem, and his latest blog post is a blueprint for making that relationship more productive and professional. Titled A Tester’s Commitments, James starts by writing:

Dear Programmer,

My job is to help you look good. My job is to support you as you create quality; to ease that burden instead of adding to it.

What follows are twelve commitments a tester should make to their programmers. They include things like:

• I provide a service. You are an important client of that service. I am not satisfied unless you are satisfied.
• I will learn the product quickly, and make use of that knowledge to test more cleverly.
• I will not carelessly waste your time. Or if I do, I will learn from that mistake.

But James is not in usual form unless he invites controversy, and that first bullet struck quite a chord with some of his readers. Testers provide a service!? Since when?

Read more…

Ho, ho, ho!  Whoa there, Blitzen– wasn’t it just Halloween?  It sure feels that way. After all, I still have two pounds of trick-or-treat candy to pretend I’m not eating.

Unfortunately, my four-year-old has already implored me to take down the skeleton and spiders hanging in the doorway because they’re going to scare away Santa.  So, rather than arguing the salient fact that Santa shimmies down the chimney versus ringing the doorbell, I’ve officially started morphing decor from the marvelous macabre to merry old Saint Nick.  Kids: 1. Mom: 0.

Nonetheless, the fact hasn’t escaped me that we’re two weeks away from Cyber Monday (November 28th), an occasion that online retailers have been planning for months.  Since summer, global brands and independent e-tailers have been testing and re-testing their mobile apps and web sites for functionality, usability, localization glitches and possible bottlenecks in site performance that could jeopardize their revenue potential.

Moreover, the ante has been upped now that the iPad and other tablets have entered the scene.  Online retailers that spent the last few years optimizing their mobile apps and porting them to additional platforms like Android, are now going through the process from scratch with tablets.  Not only are the specs non- standardized, varying significantly by manufacturer, device and network performance like smartphones.

Read more…

By now, everyone in the world of web programming has had at least some experience with HTML-5. The web is loaded with great resources that provide all the details you’d ever want to know about this new standard for structuring your webpage. But with all the information out there, some are still confused as to what all the fuss is about.

Well, I’ll tell you. Here are a few of the things that I’m most excited about with regard to HTML-5:

• New Semantic based tags instead of old div’s: Traditionally, a web developer’s life was overrun with generic divs and spans for all kind of containers in HTML. With HTML-5, there are new semantic-based tags which are container relevant to their usage. There are a number of tags introduced named <header> for header of webpage, <footer> for footer , <section> etc which are more relevant to their usage than the previous generic divs.
• No Plug-in for Video: Previously, video required some type of plug-in, like Flash, QuickTime or Silverlight to name a few. With HTML-5, we can now simply use the <video> tag – how easy is that? However, for playing video with HTML-5, the limitation is that we need to encode video into 4 different types of formats to play it consistently across the web (and more than 10 types of video formats to play it across all the mobile devices).  The reason being is that we’re in the middle of a browser war when it comes to supporting video format. Someday, the battle will be over, but not anytime soon.
• No Plugin for Audio : Similar to video, audio can now be played using <audio> tag with the help of HTML-5. Again the downside is that not all browsers support them.
• Canvas Support: Canvas support is huge deal for web developers. With the power of Canvas, they can now draw things programmatically and dynamically (on the fly) on to their screen (stage). In the past, they were dependent on languages like action script for such activities.

I can go on praising HTML-5 for its other features – like support for geo-location, offline storage and history API – but that’s not point of this blog. There are a lot of informative resources available online (like this) if you are interested in knowing more about that.

The reason I’m so interested with HTML-5 has to do with the terrific support and response from the developer community. Specifically, I’m wondering if the tester community has (or will have) the same sort of enthusiasm for HTML5.

Read more…

Ok, so it’s one of many books on the topic of testing. Still, how many crowdsourcing companies can honestly say that their community includes a published author (not to mention a top journalist in their space and former Testing The Limits guest)? Well, that’s the case here at uTest with Matt Heusser (@mheusser).

Anyway, this is a worthwhile read for two reasons:

1. Heusser knows of which he speaks, as he’s not just a pundit. This guy has lived it, running QA organizations within large and small companies.
2. Whether they’ll admit it or not, the TCO of testing is a major concern for tech execs in all industries. So any book that tackles that issue head-on is both ambitious and timely.

For those who purchase How to Reduce the Cost of Software Testing, drop us a note and let us know what you think. We’ll publish your comments in a follow-up post.

Also, we’ll see if we can wrangle an interview with the brains behind the book (Heusser and Govind Kulkarni) in the next week or so. Want us to grill them on anything in particular? Drop us a comment and we’ll put ‘em on the spot.