Testing Usability within an Agile Framework

Adapting to AgileWe’re back with another usability testing guest post from uTest UX Expert Inge De Bleecker. Last year Inge gave us a run down of the basics of UX testing and exploratory UX testing. Now, Inge, a user experience architect by trade, is back to share her tale of transitioning to an Agile process after 15 some odd years in the business.

To learn more about Inge, visit her LinkedIn profile or check out her blog.


Lean UX: User Experience on the Fast Track

A few years ago, the company I worked for ‘went Agile’. As a user experience designer, I was scratching my head. The discipline of user experience was just gaining recognition as a valid discipline, and I had spent a good bit of time educating my colleagues on how user experience activities fit in their waterfall process. Now I was facing a whole different set of challenges trying to fit user experience into the Agile process.

One of the main concerns I had with Agile was the lack of upfront planning time. In order to build a product that users want to use, it is important to understand who the target user is, and how they expect to interact with the product. Armed with that insight, an overall information structure and blueprint for the user interface is then created. All this preliminary work takes time and effort, and has to be completed before diving into the details and actual development. Fortunately, most development teams I worked with struggled with similar upfront time constraints to get their tools running and their preliminary planning completed. That bought us some much appreciated extra time.

Very soon, I came to love one particular aspect of Agile: the short sprints introduced the perfect opportunity to be iterative. Design is inherently an iterative activity, and Agile supports that very well. I quickly got into the habit of reviewing sprint deliverables, throwing in a quick usability test at times, and submitting a list of change suggestions for consideration in the next sprint planning meeting. Such quick turnaround time for making changes to a product interface had never been possible before.

Over the past few years, many organizations have moved to some version of Agile, and many user experience designers have had to adapt. The result on the user experience side is now called ‘Agile UX’ or ‘lean UX’. Some view the lean UX movement as a ‘revolution’, for better or for worse. Proponents prefer to call it an ‘evolution’. Whichever way you look at it, two interesting changes came out of this:

1. A wider acceptance of ‘quick’ work that allows and forces us to focus on results instead of process and methodology: there is no time to write elaborate plans and 60-page result reports, no time to draw pretty pixel-perfect pictures and exhaustive wireframes. Remote and unmoderated usability testing methods are becoming more widely accepted as one of the tools in the user experience toolbox.

2. Tools to enable a quick turnaround: Previously, user experience designers were forced to master complex drawing programs to create their deliverables. Those programs were not all that well-suited to the work and were time-consuming to learn and use. Today, we have a choice of tools that are low cost, have a shallow learning curve and that allow us to create rough (and not so rough) visualizations that get the ideas across.

To be clear, there is no substitute for thorough planning and clear communication of ideas and feedback. We should never use Agile as an excuse to skim over the essentials of good user experience design. As long as we keep this in mind, Agile and the lean UX evolution have helped us greatly in building more successful products.

Top Global Brands Lack Mobile

Mobile AppsYou may have seen the sweet new infographic we posted about the quality of mobile apps. And with smartphones and tablets everywhere you turn you might assume every business from mega corporations down to the mom and pop shop on the corner would have a mobile app by now – but you’d  be wrong. Digiday took a look at the mobile offerings from the top 50 global brands (as named by Interbrand) and found quite a few of them lacking. Here’s the scoop:

Digiday examined the websites of the top 50 global brands of 2011, as ranked by Interbrand, and found that 19 do not currently feature smartphone-optimized content. Those brands, which include Apple, Microsoft, GE, Nokia, Nintendo, Mercedes, BMW and Kelloggs, simply drive smartphone users to the desktop versions of their sites. …

The number of brands offering tablet-specific experiences, meanwhile, was even lower. Of the 50 brand websites examined, just two provided content tailored for the iPad. Those brands were Google and Nike.

Surprisingly, top electronics and technology companies were among the least prepared for mobile traffic. Apple, Nokia, and Microsoft, all of which sell or manufacture mobile devices or software, had neither smartphone- nor tablet-optimized sites. Despite the fact major auto manufacturers are often viewed as early adopters of the mobile channel, the sites of Mercedes and BMW were also lacking in both areas. Toyota, Honda, VW and Ford, meanwhile, all served up content tailored for smartphones, but not for the iPad.

Interestingly, Pepsi currently redirects both smartphone and tablet users to its Facebook page and appears to have no mobile content whatsoever hosted on its Pepsi.com domain. Though that approach might not be viewed as a “mobile-ready” one, the experience is at least tailored to users’ devices once they reach Facebook, owing to the social network’s support for a range of devices.

Many marketers argue a customized experience for the iPad isn’t necessary, since screen size and full-featured Safari browser are adept at rendering full desktop sites. But those experiences are primarily designed for use with a mouse and keyboard, and do not take into account the touch functionality of the iPad and other tablets. Nike’s iPad site, for example, features similar content to its desktop site but allows users to touch, swipe and interact with it in a much more intuitive way. …

Continue Reading

[Infographic] The State of Mobile App Quality: Android vs. iOS

It’s the industry’s premiere event, attended by some of the biggest names and brightest stars in the world…and it’s not the Academy Awards. I’m talking of course about Mobile World Congress, which kicks off today in Barcelona, Spain. While mobile enthusiasts convene to see what’s new and what’s next, we here at uTest decided to take at look at the current state of mobile app quality, which brings us to the following infographic. Below is an in-depth a look at the state of user satisfaction in the top two mobile ecosystems: iOS and Android.


uTest Joins the Ranks of Hot Startups on VatorTV [VIDEO]

What does uTest’s Doron Reuveni have in common with Box’s Aaron Levie, Turntable.fm’s Seth Goldstein, AirBnB’s Brian Chesky, Kaggle’s Anthony Goldbloom and Gaia’s Craig Sherman?  They’ve all been interviewed over the past year by award-winning journalist Bambi Francisco Roizen for Vator.tv.

Last week, uTest joined the ranks of these hot, innovative startups by appearing on VatorTV, one of the largest business networks dedicated to entrepreneurship, and the sister site to VatorNews, which is focused on the business and trends of high-tech entrepreneurship and innovation with 400-plus contributors.

Bambi, the CEO and founder of Vator (short for ‘innovator’), caught up with Doron to learn the ins-and-outs of uTest’s business model and what our expansion plans are for 2012 following our recent $17M D round of funding.

Gamers Help Break Kickstarter Funding Records

Gaming industry veteran Tim Schafer took up the help of Kickstarter to raise $400,000 in order to fund the creation of his company’s new adventure game, as well as film the progress of the game development for everyone to view as bonus content during the production. Within hours, Schafer’s studio, Double Fine, reached their goal, breaking the record for the most money earned in the quickest time on Kickstarter. At this current time, with 19 days remaining on Kickstarter, Double Fine has over 62,800 backers and cover $2 million in pledges.

A Kickstarter spokesperson has confirmed that Double Fine’s adventure game fundraiser now holds the company record for raising such a high amount of money in a short amount of time.

“I can confirm that there’s not been a project that has raised as much as this one in such a short timeframe, and now has more backers than any other project on the site,” the spokesperson revealed.

Kickstarter says it does not keep a running tally of finalized projects, but its listing of ‘Most Funded‘ ventures shows a number of concepts that came close to the one million dollar mark since 2009.

Ideally, this is how video games would be made. People in the community, and people that genuinely love what they do in the video game industry, coming together to build something everyone can call their own. The only thing more exciting than a new old-school point and click adventure game from Tim Schafer is the outlook on how this campaign may impact the traditional game publishing paradigm going forward. You can keep an eye on their progress or contribute to the project over on Double Fine’s Kickstarter page.

Testing Roundtable: What’s the Biggest Weakness in the Way Companies Test?

This month, in place of our standard Testing the Limits interview, we decided to hit up a few of our past guests for a “testing roundtable” discussion. The topic: What is the biggest weakness in the way companies test software? Below are some extremely insightful answers from testing experts Michael Bolton, James Bach, Noah Sussman, Dan Bartow, Rex Black, Jim Sivak and Cem Kaner. Enjoy!


Michael Bolton, Principal at DevelopSense:

So far as I can tell, most companies treat software development as implementation of highly idealized business processes, and they treat testing as an exercise in showing that the software models those processes in a way that’s technically correct. At the same time, companies treat the people who use the software as an abstraction. The consequence is that we’re creating software that delays and frustrates the people who use it or are affected by it. When testing is focused almost entirely on checking the functions in the software, we miss enormous opportunities to learn about the real problems that people encounter as they go about their business. Why are testers so often isolated from actual end-users?

Today I was traveling through the airport. When I checked in using the online service, I had accidentally noted that I’d be checking two bags, but I only brought one with me. In addition, my flight was cancelled, and I had to be put on a later flight. The customer service representative could get me onto that flight, but she had serious trouble in printing a boarding pass associated with only one bag; apparently there was a warning message that couldn’t be dismissed, such that her choices were to accept either three bags or none at all. It took fifteen minutes and two other representatives to figure out how to work around the problem. What’s worse is that the woman who was trying to help me apologized for not being able to figure it out, as if it were her responsibility. Software development organizations have managed to convince our customers that they’re responsible for bugs and unforgiving and unhelpful designs.

The success of a software product is only partly based on how it handles the happy path. That’s relatively easy to develop, and it’s relatively easy to check. Real testing, to me, should be based on investigating how the software allows people to deal with what we call “exceptions” or “corner cases”. That’s what we call them, but if we bothered to look, we’d find out that they were a lot more common than we realize; routine, even. Part of my vision of testing is to include a new discipline in which we do significant field research and participant observation. Instead of occasionally inviting customers to the lab (never mind sitting in the lab all by ourselves), we testers—and our organizations—could learn a lot through direct interaction with people who use the software every day; by close collaboration with technical support; and by testing rich and complex scenarios that are a lot closer to real life than simplified, idealized use cases.


James Bach, Author and Consultant, Satisfice:

There is a cluster of issues that each might qualify as the biggest weakness. I’ll pick one of those issues: chronic lack of skill, coupled with the chronic lack of any system for acquiring skill.

Pretty good testing is easy to do (that’s partly why some people like to say “testing is dead”– they think testing isn’t needed as a special focus because they note that anyone can find at least some bugs some of the time).

Excellent testing is quite *hard* to do.

Yet as I travel all over the world, teaching testing and consulting in testing organizations, I see the same pattern almost *everywhere*: testing groups who have but a vague, wispy idea what they are trying to do; experienced testers who barely read about and don’t systematically practice their craft beyond the minimum needed to keep their employers from firing them; testers whose practice is dominated by irrational and ignorant demands of their management, because those testers have done nothing to develop their own credibility; programmers who think their automated checks will save them from disaster in the field.

How does one learn to test? You can’t get an undergraduate degree in testing. I know of two people who have a PhD in testing, one of whom I admire (Meeta Prakash), the other one is, in my view, an active danger to himself and the craft. I personally know, by name, about 150 testers who are systematically and diligently improving their skills. There are probably another several hundred I’ve met over the years and lost touch with. About three thousand people regularly read my blog, so maybe there are a lot of lurkers. A relative handful of the people I know are part of a program of study/mentoring that is sanctioned by their employers. I know of two large companies that are attempting to systematically implement the Rapid Testing methodology, which is organized around skill development, rather than memorizing vocabulary words and templates. Most testers are doing it independently, however, or even in defiance of their employers.

Yes, there is TMap, TPI, ISTQB, ISEB, and many proprietary testing methodologies out there. I see them as crystallized blobs of uncritical folklore; confused thinking about testing frozen in place like fossilized tree sap. These models and procedures have been created by consultants and consulting companies to justify themselves. They neither promote skill or require skill. They promote what I call “ceremonial software testing” rather than systematic critical thinking about complex technology.

Just about the best thing a tester can do to begin to develop testing skill in a big way is not to read or study any test methodology. Ignore vocabulary words. Toss aside templates. No, what that tester should do is read Introduction to General Systems Thinking, by Gerald M. Weinberg. Read it all the way through. Read it, young tester, and feel your mind get blown. Read it, and meditate on its messages, and do the exercises it recommends, and you will find yourself on a new path to testing excellence.


Noah Sussman, Technical Lead, Etsy:

A surprising number of organizations seem to dramatically underestimate the costs of software testing.

Testability is a feature and tests are a second feature. Having tests depends on the testability of an application. Thus, “testing” entails the implementation and maintenance of two separate but dependent application features. It makes sense then that testing should be difficult and expensive. Yet many enterprise testing efforts do not seem to take into account the fact that testing an application incurs the cost of adding two new, non-trivial features to that application.

There also seems to be a widespread misconception that testing somehow makes application development easier. In fact the opposite is true.

If I may mangle Kernighan: testing is much more difficult than writing the code in the first place. To implement testability and then write tests, one needs first to understand the architecture of the application under test. But testing also requires doing hard things — like input partitioning and path reduction — that are beyond the scope of the application. The reality is that to get good tests, you’re going to have to ask some of your best people to work on the problem (instead of having them work on user-facing application features). Yet many organizations seem not yet to have recognized this.


Continue Reading

7 Tips To Make A Bring-Your-Own-Device Policy Safe For Work

Bring Your Own DeviceThe growing “bring your own device” trend has some enterprise companies wringing their hands with worry. How do you keep employees happy while still protecting company data? Well, InformationWeek has seven tips to help companies navigate this new frontier (I threw a few notes in as well):

1. Create Strong Security Policies

While it might sound basic, having mobile device security policies in place is a necessary first step. … An organization in a highly regulated industry may specify that all data stored on employees’ mobile devices, as well as any removable media used with those devices, be encrypted. Businesses in other industries, however, may think that approach is overkill.

2. Apply Existing Security Policies To Mobile Devices

When crafting mobile device security policies, carry through existing policies. For example, if you require that passwords for accessing the corporate network have 15 characters, mixing uppercase, lowercase, and at least one symbol, then the same should be true for any mobile device that’s allowed to connect to the corporate LAN. … Also weigh whether Bluetooth file-sharing will be allowed for mobile devices, and if jailbroken devices should be blocked from accessing the network altogether.

3. Enforce Security Policies

The next step is to enforce your organization’s policies, typically by using mobile device management (MDM) tools [Note: The mobile device management link leads to another great article on protecting your company while allowing employees to use their own devices. It's definitely worth a read]. Regardless of the approach selected, without enforcement, employees will see your mobile security policies as optional, especially you have a bring your own device (BYOD) to work policy. [Another Note: 21% of employees in a recent survey said they don't know their company's IT security policies. 33% said they don't always follow them. Check it out at InfoSecurity >>>]

Continue Reading

Video Game Teaches You How to Code

The biggest hurdle to becoming a proficient coder has always been a lack of simulated, first-person shooting. You know this to be true. That’s about to change with the launch of Code Hero, a video game that teaches you how to code. Here are the details from Wired.com:

The game is a co-op first-person science shooter where you use a code gun to manipulate code. Your code gun can copy code like new items and fire it like ammunition to do new things. You can edit new code to do anything you can imagine. You’ll learn how to blast the enemy, manipulate the world, and build structures creatively to create the games of your dreams and recruit an army of coders to save the world from rogue AI.

In all seriousness, this looks like a fun way to help people of all ages become a well-versed in the art of coding. As it happens, the game is currently in beta and can be downloaded (and tested!) with a modest KickStarter donation. As testers who “sometimes” work with buggy code, I would consider this a wise investment.

Anyway, I suppose this would a good time to remind you that we currently have our own video game – Bugbusters – available on our Facebook page. It does not involve guns, it will not teach you how to test software, but you could win an iPad2.

Check it out >>>

10 Tips To Protect Against A DDoS Attack

Group AttackIf you follow tech news you’ve undoubtedly seen multiple stories lately of websites falling to DDoS attacks. Most of these attacks have been by Anonymous and targeted government, big media or SOPA/PIPA/anti-piracy supporters’ sites. But their actions have also begun inspiring like-minded hackers who fancy themselves “hacktivists.” Whether these hackers are taking your site down in protest, to make a point or for financial gain doesn’t really matter to you, because in the end, your site is still down.

With this growing trend and the increase attacks on consumer sites it’s important to know what steps you can take to prevent a DDoS attack from effecting your business. With that in mind, InformationWeek put together a list of “10 Strategies To Fight Anonymous DDoS Attacks.” Here’s what they suggest:

1. Know You’re Vulnerable

One lesson from the use of DDoS by Anonymous–as well as its sister hacktivist group LulzSec–is that any site is at risk. That’s not meant to sound alarmist, but rather simply to acknowledge that the hacktivist agenda can seem random, at best.

2. DDoS Attacks Are Cheap To Launch, Tough To Stop

Hacktivists can quickly crowdsource “5,600 DDoS zealots blasting at once,” as Anonymous boasted on Twitter, to take down the websites of everyone from the FBI and the Justice Department to the Motion Picture Association of America and Recording Industry Association of America. “DDoS is to the Internet what the billy club is to gang warfare: simple, cheap, unsophisticated, and effective,” said Rob Rachwald, director of security strategy of Imperva, via email.

3. Plan Ahead

Stopping DDoS attacks requires preparation. If attacked, “folks that don’t take active measures to ensure the resilience of their networks are going to get knocked over,” said Roland Dobbins, Asia-Pacific solutions architect for Arbor Networks, via phone.

Continue Reading