Software Testing Blog

Over the last few months, I have had the privilege of speaking with uTesters from all over the world. A large chunk of these conversations centered on their uTest experience, as well as new ways they can benefit from being part of a diverse global community of software testers. One of the main themes that stood out to me was the vast array of knowledge and experience that many tester had under their belts, and no tester having the same footprint of theoretical and practical knowledge.

And given the propensity of uTesters to share their knowledge with each other, it seems natural to provide more peer-to-peer learning opportunities. The idea was not only to provide learning resources, but also to give various experts a chance to share their knowledge with others. Thus was born a new collaborative learning initiative called the uMentor Program.

What is it?

The uMentor Program is a resource designed to foster collaborative learning within the uTest community by identifying experts in our community, called uMentors, and giving them an avenue to address questions from the larger community. This program will help new testers get ramped up and learn the skills they need to become gold testers, as well as help seasoned testers learn new skills and expand upon their testing knowledge.

How does it work?

Every week we will kick off a new topic (pre-determined by previous requests). A uMentor will post a topic, addressing the high-level aspects of this topic, and then anyone can ask follow-up questions. The topic will lock after two weeks, but will remain available for you as a resource. In addition to Q&A posts on the forums, we will also host a monthly webinar based on a pre-determined topic where a uMentor will deliver a live presentation and end with ample time for questions and answers.

For more information about how to get involved, check out the uMentor post on the uTest Forums. If you are not a member of our software testing community yet and want to contribute and/or participate, feel free to sign up as a uTester here before gaining access to the uMentor Program and other free testing resources.

What’s the next hurdle to overcome after you establish a strong product-market fit? That’s the topic I’ll be discussing next week as part of the Vilna Shul Speakers Series. I’ll be presenting alongside Brian Halligan (CEO and Co-founder of Hubspot), Diane Hessan (CEO of Communispace) and Gail Goodman (Chairman and CEO of Constant Contact). As you can see, I’m in good company. Pardon the pun.

It’s often said that product-market fit is the most important factor in the success or failure of a new startup. And I agree to the extent that you can’t succeed without it – and many startups never achieve it. Establishing a strong product-market fit was a major challenge for uTest. The software testing industry was entrenched, mature and many people (including us in our weaker moments) were skeptical that crowdsourcing could play a role. I think we all know how that turned out.

As challenging as this step may be, it pales in comparison to what comes next: securing funding to take bigger bets; acquiring & retaining talent; and identifying growth vectors to accelerate growth. These things are not easy.

So, for those who are interested in achieving startup success, but won’t be able to attend, I wanted to share a few quick pieces of advice on what to do after you determine a product-market fit. Here’s a portion of what I hope to share with the panel:

Read more…

We’re back with another usability testing guest post from uTest UX Expert Inge De Bleecker. Last year Inge gave us a run down of the basics of UX testing and exploratory UX testing. Now, Inge, a user experience architect by trade, is back to share her tale of transitioning to an Agile process after 15 some odd years in the business.

To learn more about Inge, visit her LinkedIn profile or check out her blog.

***********************

Lean UX: User Experience on the Fast Track

A few years ago, the company I worked for ‘went Agile’. As a user experience designer, I was scratching my head. The discipline of user experience was just gaining recognition as a valid discipline, and I had spent a good bit of time educating my colleagues on how user experience activities fit in their waterfall process. Now I was facing a whole different set of challenges trying to fit user experience into the Agile process.

One of the main concerns I had with Agile was the lack of upfront planning time. In order to build a product that users want to use, it is important to understand who the target user is, and how they expect to interact with the product. Armed with that insight, an overall information structure and blueprint for the user interface is then created. All this preliminary work takes time and effort, and has to be completed before diving into the details and actual development. Fortunately, most development teams I worked with struggled with similar upfront time constraints to get their tools running and their preliminary planning completed. That bought us some much appreciated extra time.

Very soon, I came to love one particular aspect of Agile: the short sprints introduced the perfect opportunity to be iterative. Design is inherently an iterative activity, and Agile supports that very well. I quickly got into the habit of reviewing sprint deliverables, throwing in a quick usability test at times, and submitting a list of change suggestions for consideration in the next sprint planning meeting. Such quick turnaround time for making changes to a product interface had never been possible before.

Over the past few years, many organizations have moved to some version of Agile, and many user experience designers have had to adapt. The result on the user experience side is now called ‘Agile UX’ or ‘lean UX’. Some view the lean UX movement as a ‘revolution’, for better or for worse. Proponents prefer to call it an ‘evolution’. Whichever way you look at it, two interesting changes came out of this:

1. A wider acceptance of ‘quick’ work that allows and forces us to focus on results instead of process and methodology: there is no time to write elaborate plans and 60-page result reports, no time to draw pretty pixel-perfect pictures and exhaustive wireframes. Remote and unmoderated usability testing methods are becoming more widely accepted as one of the tools in the user experience toolbox.

2. Tools to enable a quick turnaround: Previously, user experience designers were forced to master complex drawing programs to create their deliverables. Those programs were not all that well-suited to the work and were time-consuming to learn and use. Today, we have a choice of tools that are low cost, have a shallow learning curve and that allow us to create rough (and not so rough) visualizations that get the ideas across.

To be clear, there is no substitute for thorough planning and clear communication of ideas and feedback. We should never use Agile as an excuse to skim over the essentials of good user experience design. As long as we keep this in mind, Agile and the lean UX evolution have helped us greatly in building more successful products.

It’s the industry’s premiere event, attended by some of the biggest names and brightest stars in the world…and it’s not the Academy Awards. I’m talking of course about Mobile World Congress, which kicks off today in Barcelona, Spain. While mobile enthusiasts convene to see what’s new and what’s next, we here at uTest decided to take at look at the current state of mobile app quality, which brings us to the following infographic. Below is an in-depth a look at the state of user satisfaction in the top two mobile ecosystems: iOS and Android.

*******

What does uTest’s Doron Reuveni have in common with Box’s Aaron Levie, Turntable.fm’s Seth Goldstein, AirBnB’s Brian Chesky, Kaggle’s Anthony Goldbloom and Gaia’s Craig Sherman?  They’ve all been interviewed over the past year by award-winning journalist Bambi Francisco Roizen for Vator.tv.

Last week, uTest joined the ranks of these hot, innovative startups by appearing on VatorTV, one of the largest business networks dedicated to entrepreneurship, and the sister site to VatorNews, which is focused on the business and trends of high-tech entrepreneurship and innovation with 400-plus contributors.

Bambi, the CEO and founder of Vator (short for ‘innovator’), caught up with Doron to learn the ins-and-outs of uTest’s business model and what our expansion plans are for 2012 following our recent $17M D round of funding.

For those of you who only read the uTest blog for the funny pages, here you go:

Deep thoughts today on the uTest Blog. Out of curiosity, I did a little research on the most difficult bugs to uncover and found an interesting answer: the Heisenbug. For those unfamiliar with this term, here’s a good definition from Wikipedia:

Heisenbug is a whimsical computer programming jargon term for a software bug that seems to disappear when one attempts to study it. The term is a pun on the name of Werner Heisenberg, the physicist who is commonly associated to the observer effect of quantum mechanics, which states that the act of observing a system inevitably alters its state.

How would such a bug manifest itself in the real world? Here’s one example:

Heisenbugs occur because common attempts to debug a program, such as inserting output statements or running it in a debugger, usually modify the code, change the memory addresses of variables and the timing of its execution.

One common example of a Heisenbug is a bug that appears when the program is compiled with an optimizing compiler, but not when the same program is compiled without optimization (as is often done for the purpose of examining it with a debugger). While debugging, values that an optimized program would normally keep in registers are often pushed to main memory. This may affect, for instance, the result of floating-point comparisons, since the value in memory may have smaller range and accuracy than the value in the register. In these cases, the bug is in the optimizing compiler and not in the program.

Of course, the idea of a disappearing software bug leads one to ask: if a software bug cannot be found, does it really exist?

Read more…

You can kiss productivity goodbye for the next four weeks, because we have just launched BugBuster – a new gaming contest on Facebook!

Here are the details:

• In order to participate, you must “like” the uTest Facebook page
• Next, click on the Bugbuster game page, “Go to app,” and finally “Play” to begin
• Contestants have until March 7^th to participate (unlimited number of plays)
• Awards will be given to the top 3 highest scorers:
  • 1^st Place: iPad 2
  • 2^nd Place: Digital camera
  • 3^rd Place: Digital camera

We look forward to your participation in this contest and invite you to dethrone the current highest scorer! Remember, you have unlimited attempts until March 7^. Winners will be announced shortly thereafter.

Happy bug catching!

Whether you’re on an in-house QA team or an independent tester, you’ll probably recognize these 13 distinct programmer personalities (compiled by PCWorld). Maybe you love some, maybe you hate others but they’re all out there and they probably won’t be going away anytime soon. So read on, and leave us a comment to let us know which type of programmer is your favorite from a testing standpoint.

1. The Underdocumenter
The Underdocumenter will go to any length to avoid being shackled by management’s foolish requirement to write text about that function. The droll ones will create functions like queryDatabase, then add a comment that says, “Queries database.”

Car: Vespa
Relationship status: Living with the same person for 15 years without getting married because they don’t want to fill out the forms
Household chore: Rewiring the house without labeling the breakers
Role model: Guy who hid the Ark of the Covenant before “Raiders of the Lost Ark”
Pet: “Around here somewhere.”
Favorite programming construct: Lambda
Drink: Anything with an “XXX” on the bottle

2. The CYA Specialist
If you’re lucky, your CYA Specialist will be a frustrated novelist who is happy to inject a pun or two into a boring pile of code. But the worst kind is the one who lords their documentation over others during code reviews. If a bug appears, the CYA Specialist says it was a limitation that was “well-documented in the 17th paragraph of the method’s comment.”

Car: Stack of Chilton manuals
Relationship status: Married to a 48-page prenuptial agreement
Household chore: Relabeling the spice rack
Role model: Wikipedia editor of the year
Pet: “Come over to see the photo montage of Scrappy that used to be just a wall.”
Favorite programming construct: The comment block
Drink: Triple-filtered water

Read more…

Every once and awhile, when there’s nothing topical to blog about, I decide to go back in time and focus on a software testing classic. Today is one of those days.

With that in mind, I wanted to draw your attention to Classic Testing Mistakes by Brian Marick. Whether you’re a tester or manager, experienced veteran or wide-eyed newbie, this 25-page article outlines some of the most classic testing mishaps and offers valuable tips on how to avoid them.

So what are some classic testing mistakes? One would be not reading this document in its entirety. As for the others, here are a few clips I found interesting. Note the bold titles are mine, everything in italics in Brian’s.

Mistake: Testers Not Responsible for Usability
If usability problems are not considered valid bugs, your project defines the testing task too narrowly. Testers are restricted to checking whether the product does what was intended, not whether what was intended is useful. Customers do not care about the distinction, and testers shouldn’t either.

Mistake: Misunderstanding the Role of “QA”
A first major mistake people make is thinking that the testing team is responsible for assuring quality. This role, often assigned to the first testing team in an organization, makes it the last defense, the barrier between the development team (accused of producing bad quality) and the customer (who must be protected from them). It’s characterized by a testing team (often called the “Quality Assurance Group”) that has formal authority to prevent shipment of the product. That in itself is a disheartening task: the testing team can’t improve quality, only enforce a minimal level. Worse, that authority is usually more apparent than real.

Mistake: Bad Timing on Load Testing
Putting stress and load testing off to the last minute is common, but it leaves you little time to do anything substantive when you discover your product doesn’t scale up to more than 12 users.

Read more…