Do Functional Testers Make Good Usability Testers?

Would you trust a plumber to design and decorate the interior of your new home? How about an auto mechanic to paint your car? Or a dermatologist to perform your root canal? If you answered no to any of these questions, then I would argue just as strongly that functional testers should not be handling usability testing for your software.

To be absolutely clear, there are some talented functional testers who are gifted in usability testing as well. However, this is typically the exception. The point of this blog post is not to minimize these exceptions, but rather to highlight the differences between these two testing types and point to the necessity of having one set of experts test your software for functionality, and another set of experts test for usability.

So why do we need separate experts? Here are several noteworthy observations:

  • Different fields of study: In general, usability testing experts come from non-technical fields of study – psychology, cognitive science, human behavior, marketing, etc.
  • Different skills required: In general, functional testers have more in common with engineers and scientists while usability testers have more in common with artists and psychologists
  • Different focus: In general, functional testers focus on “how can I break this” whereas usability testers focus on “how intuitive is this for the end customer”
  • Different persona: In general, functional testers do not need to fully understand how an end user would use the software whereas the usability tester needs to study and emulate end users
  • Different departments in the workplace: In general, usability testing lives in the product management and marketing department while functional testing lives in the engineering department

Given these differences, in particular regarding fields of study and skill sets, I would recommend against entrusting your usability testing to the typical functional tester (the opposite is true as well – entrusting your functional testing to the typical usability tester could be disastrous). There is a particular art and science to usability testing that requires an entirely different approach and mindset. So from a company’s perspective, it’s best to separate these roles and recognize their distinct value propositions.

Continue Reading

Uruguay Launches Collegiate Software Testing Program

The Uruguay IT Chamber has launched a new university program for software testing.  This is big news for the world of software testing, as it not only legitimizes the professional tester but will help draw attention (and projects) to the importance of testing. has a nice write up on the news release, but it’s particularly interesting to see that there are three tiers of testing, as follows:

Software Tester: The ideal program for entry-level, junior, and career-switching software testers.  Successfully completing this credential will give you the knowledge and experience most employers expect from testers with 1-2 years of on-the-job experience – effectively enabling you to start your career needing only to acclimate to the specific expectations of the employer and demonstrate your skills at work before being acknowledged as a mid-level tester.

Software Testing Professional: For testers with several years of testing experience on-the-job who are looking to make the jump from mid- to senior-level, this program is designed to teach high-quality individual contributors how to be effective technical leaders within their testing organization.  Effectively giving the tester the new skills they need, in addition to their existing hands-on testing knowledge, to prepare them for the additional responsibilities of a technical testing manager or of a manager of small to mid-sized testing projects.

Software Testing Leader: This final step in the program has been designed for senior-level testers who desire to be successful managers or directors of corporate testing programs.  Frequently the most challenging step for career software testers is to transition from being a technical leader to a management role focused on the interface between quality testing and executive-level business value.  Successful completion of this stage of the program will give the student the tools they need to make the jump from technical leader to manager – opening the door to further advancement to positions like “Director of Testing Services” or “VP of Software Product Quality”.

Full article after the break.

Continue Reading

The Silver Lining to Motorola’s Comments on Android

Over the past week, there’s been some hub bub over comments made by Motorola’s CEO Sanjay Jha.  According to IDG News Service, Jha “blamed the open Android app store for performance issues on some phones,” based on his statement: “Of all the Motorola Android devices that are returned, 70 percent come back because applications affect performance.”

Even though Motorola formally stated today (see MoCoNews article) that Jha’s comments were essentially misconstrued and didn’t accurately reflect his intentions, the issue has remained a lightning rod for debate.

But for those of us in the software testing community, there’s a truly, positive message embedded in this issue:  Motorola was validating the critical importance of QA testing in the app development process.  

After all, consider Jha’s statement that, “one of the good and problematic things about Android is that it’s very very open. So anyone can put applications, third-party apps, on the market without any testing process….For power consumption, CPU utilization, some of those things, those applications are not tested. We’re beginning to understand the impact that has.”

For professional software testers, that confirms how important our work is, and actually suggests that the scope of mobile testing should be expanded.

Essentially, Jha wasn’t really referring to functional testing.  Or testing exclusively in the “clean and ideal” conditions of a lab environment.  Instead, he was describing the need for usability testing in the real-world to subjectively examine how apps and devices perform in live conditions and affect the user experience.  For instance, did the app run sluggishly?   Did it seriously tax the battery life?  These are vital questions, particularly for apps heavy on audio and video. 

At the end of the day, consumers are unlikely to differentiate whether their frustration over poor performance is caused by the smartphone or the app…or the interaction of both.  They just want to have a great experience with their new mobile “toy” or get their work done. 

Because if there isn’t enough testing on every device that the app is developed for, then (as Jha said) the smartphone gets returned and everyone– including the app publisher–loses out.

Test Your Web Apps – Express for Web

Two months ago, we announced uTest Express – an easy and affordable way for startups and young companies to test mobile apps. So far, it’s been a huge success, and today we’re pleased to announce a big update: uTest Express for Web. Now you can quickly and easily have your web application or web site tested by a team of professional testers under real-world conditions.

Like the mobile version, Express for Web helps you validate your app by providing you with both bugs and feedback from real people testing on real computers. With Express for Web, we’ll make sure there’s a sold cross-section of today’s most popular browsers and operating systems to give you a good feel for how your app or site behaves in today’s complicated browser landscape.

So how much does this awesome testing cost? Well, plans start at $499, but to celebrate the launch of Express for Web we’re discounting prices for the Bronze and Silver packages by 50%. Between now and the end of June, you can have your web or mobile app tested for as low as $249.

So what do you get for that amount? Well how about access to handpicked members of the uTest community from North America who best match your testing requirements. Web testers will be armed with a wide variety of browsers while mobile testers will test your app on real devices using real carriers. In both cases, your app is tested professionally by real people who provide real-world testing results and expert feedback that isn’t possible with emulators, simulators or remote access.

At the conclusion of each project, you’ll receive a list of well-documented bugs, including screenshots and videos with steps to reproduce them. You’ll also receive expert feedback from the testers about the application – including app ratings and feedback for interface design, usability, app performance and more. To learn more about how it works, watch this brief Product Tour.

Here are a few key features of the service:

  • Cost-Effective: With three testing packages available, uTest Express offers mobile and web app testing to fit any startup budget;
  • Easy-to-Use: A simple interface walks customers through a series of questions to identify their testing needs and facilitate the creation of the testing project;
  • Professional Testers: Testers are selected from our community of 35,000+ professional testers;
  • Accessible 24/7: Customers can access their testing projects on-the-go, using uTest’s downloadable app for the iPhone and iPad, or through the traditional browser-based interface.

So if you need iPhone app testing, Android app testing, iPad app testing, web app testing, and more, check out uTest Express.

Advice for Testers: When Your Client is on Holiday

When you’ve been testing for a while, you often latch on to a single client who is your main source of employment. They know that you do a good job, and you might have become somewhat of a ‘specialist’ in the areas that concern the client. In some months, 100% of your paycheck might be from hours spent working for this client.

But what happens when your primary client is on hiatus? Let’s say that they might be having a light month, or the work that the client generates comes in cycles, due to the nature of their industry. What do you do at that point, when your cash cow isn’t in the barn?

Well the simple answer is ‘find other work’, but it isn’t that simple in reality, now is it? The reason that certain testers gravitate towards special clients is because those clients pay well, and the tester has a very specialized set of skills. If you find other work, it’s most likely going to be less reliable hours and for less money.

My first piece of advice is: Think like a bear. A bear gathers resources and fattens up during the plentiful months, and is prepared for those lean winters. In your case, don’t spend your contracting money just because you have it. Instead, start pinching pennies and saving money a couple of months before any expected ‘lean’ period. Make sure that you always have a couple months worth of income in a savings account, just in case the well runs dry.

Secondly, when looking at replacement work, err on the side of having too much work rather than too little. For example, if you have the option of picking up a 6 week project to fill your 4 week downtime, take it. Yes, it might mean that you’re working double time for 2 weeks, dropping all social and recreational activities for that period. But take the contract, grit your teeth, and get it done. You need to cover the gap in your income, and you can’t afford to turn a project down because of a short overlap.

Continue Reading

Vote for uTest to Win the TiE50 Award!

Besides the fact that our CEO Doron Reuveni rocked it at the Boston Marathon, beating his previous year’s record by 3 minutes, we had even more great news to celebrate last week:  we found out that uTest was selected as a finalist for the 2011 TiE50 award by the Silicon Valley chapter of TiE!

The TiE50 is the premier annual awards program for thousands of technology startups worldwide.  Every year, the winners in each of five categories (ranging from software to cleantech innovations) are announced at the TiEcon event, which takes place this year in Santa Clara, CA on May 13-14th.  It’s one of the world’s largest conferences for entrepreneurs to meet and greet with venture capitalists, angels, and investors.

But we need your help!  As part of the award process, there’s a poll asking people to vote for their favorite company among the finalists (in our category, there are 43).

If you’re a fan of uTest, please vote for us by Wednesday, April 27.  It only takes 1 minute (we swear…).  Here’s how:

  1. Visit the TiE50 web site:
  2. Scroll down and click on uTest.
  3. On our profile page, enter your email address in the ID box and click the green THUMB’S UP button.
  4. That’s it!

You can also help us spread the word by retweeting or sharing this post.

Thanks for your support!!

uTest Partners with Mozilla to Build Test Case Management System

Test Case Management can be trickier than herding cats.  If you work for a small to mid-sized company and everyone’s counting on you to QA the apps to glowing perfection (no pressure, right?), then you’ve clearly demonstrated you’re a supremely organized person.  It’s in your DNA.

Yipppie kay yay

If it weren’t, you’d probably be huddled in a corner, glassy-eyed and shivering like a chihuahua from the stress of managing such a complex system.  The fragmentation inherent in managing the software testing process is no small affair.  You’re tracking hundreds (thousands?) of test cases in various stages of progress for multiple products, on scores of devices, carriers and platforms, possibly in countries worldwide…with test results feeding in from several sources.  Phew!

But no matter how organized you are, it can be tough to scale your test case library to meet the needs of your growing organization.

That’s why uTest is partnering with industry-leader Mozilla to develop an inexpensive and easy-to-use test case management solution.  When it launches in a couple months, the new offering will allow startups and mid-size ventures to easily create, manage and execute their test case libraries…without the showstopping price associated with many of the enterprise-level tools on the market today.

uTest’s CMO, Matt Johnston, explains:  “The test case management space is highly fragmented, and robust tools have been historically available only to larger enterprises with big QA budgets.  By combining Mozilla’s experience launching popular, industry-shaping products, with uTest’s deep understanding of the software testing space, this partnership will produce an exceptional test case management system that’s built from the ground up for the modern challenges faced by today’s engineering organizations.”

The forthcoming product will be an open source design and made available as a free download from Mozilla. Additionally, uTest will offer a hosted implementation that will be fully integrated with its on-demand testing platform and community of 35,000+ testers. The uTest implementation will support distribution of test cases to a company’s in-house testers, outsourced testers or selected testers in the uTest community.

The Mozilla implementation also allows the distribution of test cases to volunteer testers in the Mozilla quality community.  Additionally, with an open API, outside developers will also be able to easily integrate their own tools and systems with either the uTest or Mozilla implementations.

So stay tuned….  And let us know your thoughts:  Does it sound like uTest and Mozilla are on the right track to simplifying test case management for you?

uTest Signs The Declaration of Independence From Defects @ QUEST Boston

Well we certainly had a wicked good time at QAI QUEST (Quality Engineered Software & Testing Conference), this year on our very own soil, the “Hub of the Universe” — Boston! From the uTest tutorial on “Everything You Need To Know About Mobile App Testing,” to the amazing reception at the Seaport Hotel where we played human scrabble, to the eclectic expo, to Doron’s presentation on “The Top 10 Disruptive Trends Altering Testing,” to signing the hilarious Declaration of Independence from Software Defects — Beantown provided the ideal setting for this testing pahhty! See more pics above.

And of course we can’t let you go without sharing some insights from QUEST on today’s hottest topics and emerging trends in software QA and testing. As a bonus, the QUEST Magazine was distributed to all registered attendees and provided everyone with some great, original articles on testing. I’ve included it below. Check out eBay’s Jon Bach’s article on Transition and Exploration, uTest’s Doron Reuveni’s article on The Great Quality Challenge for Mobile Apps, HP’s Rafal Los’ article on Security and Requirements, and more! Complete magazine [pdf] after the bump.

Continue Reading

Is Software Testing Really Needed?

This question was recently posed in a programmer’s Q&A post on StackExchange, and it inevitably raised a few curious eyebrows here at uTest. Diving straight to the point, the simple and unmistakable answer is YES – software testing is needed, always was, and always will be. But in order to express a more nuanced and substantial response, let’s analyze the principle opinion of naysayers: “if we would only develop software with care, we wouldn’t need testing at all.”

Let’s run with this for a minute to see if there’s validity to this statement. After all, we have all heard of the saying that goes, “an ounce of prevention is worth a pound of cure.” There’s no doubt that if software is developed without defects in the first place, there wouldn’t be a need for subsequent testing. But in a world where even simple mobile apps on your smartphone crash from time to time, how could we possibly expect software developers to produce error-free code for higher complexity applications (e.g., dependencies of the software with various operating systems, languages, or networks)? So unless you know of software developers who defy the laws of this world, there is no way around error-free code for most of the software that you and I interact with on a daily basis.

Now that we’ve established the need for software testing, let’s take a look at several noteworthy comments from this Q&A post:

  • Testing is necessary – software is written by people; people are imperfect and make mistakes. Therefore, testing is needed as it brings balance and perspective. Would you jump on a plane in which the pilot control software is plagued with the intermittent blue screen of death?
  • Testing is necessary – for the same reason that a chef tastes his food while cooking it. In other words, the software development lifecycle calls for a minimum of a dash of unit testing and a pinch of systems testing.
  • Testing is necessary – because not matter how good you are, you can’t think of everything. And even if you can think of most things, you certainly don’t have the means to optimize your code to work seamlessly in real-world environments, where the permutations and combinations of dependent factors can cripple your coding effectiveness.

Continue Reading

Game On! BigDoor Tests Game Mechanics API With uTest

BigDoor is truly changing the game by providing gamification services to web publishers, marketers and developers around the world. Based in Seattle, BigDoor helps companies use game mechanics (checkins, badges, levels, point systems, etc.) to increase user engagement, loyalty and monetization.

In need of blackbox and exploratory testing of their newly developed API, CTO Jeff Malek realized that uTest offered a very flexible solution for getting professional QA assistance, on-demand.

“At BigDoor testing is core to our culture and part of our DNA,” said Malek. “uTest has been incredibly valuable in helping change the way we work. When our guys are sleeping, testing is still getting done and that’s a big weight off our shoulders.”

Continue Reading