Software Testing Blog

Our first-ever “Bug of the Month” competition came to a close on August 29 – and what a month it was! More than 4,000 – that’s right 4,000 – people joined uTest on Facebook to partake in the “testivities.” And just as we’d hoped, we received a lot of interesting bugs, error messages and easter eggs over the last few weeks — ranging from weird to wacky to scary, and everything in between. It all went according to plan. Excellent.

Who knew that this kind of social contest would get people so fired up (not to mention so competitive)? The contest really heated up in the final weeks, as the number of “likes” climbed higher and higher with each passing day. And similar to politicians in a “too close to call” election, participants hit the campaign trail hard in the final days, trying to convince the app testing world that their entry deserved to win.

As a result of  this overwhelming response to our inaugural contest, we decided to up the ante and add two extra prizes into the mix. So, after 4, 406 “likes” and a collection of captured bugs, we’re excited to announce the winners:

• In 1st place was Abagail Gm from the Philipines with 1844 “likes.” She walks way with a new iPod Touch for her efforts.
• Manas Mehta from India takes the 2nd prize, a trendy iPod Nano for the bug with the second highest number of “likes.”
• And the 3rd prize goes to Amos Naor for our Judge’s Choice Award (the most interesting and thought-provoking easter egg– and one we had never seen before)  For unearthing this gem, Amos will receive a stylish iPod shuffle.

Plans are already underway for the next app-related contest, so be sure to connect with us on the uTest Facebook page. In the meantime, keep your eyes pealed for the next Bug of the Month.

(The warped & witty writers over at Blonde 2.0 contributed to this post)

Apps! Apps! And more apps! As the summer starts winding down here at uTest, we’ve been able to take a step back and a closer look at the big trends emerging all around us. What has been most apparent is the tremendous spike in mobile app testing needs. From top marketing agencies to retail giants to social gaming startups, our customers are developing more mobile apps to grow (or define) their businesses than ever before.

According to Game Developer Research, 25% of game developers are now making mobile games – that’s up from a mere 12% in 2009!

In addition, a survey conducted by iGR found that more than half (53%) of US mobile developers are building apps for Apple’s iPhone OS. BlackBerry was the next most popular, followed by Android and Windows Mobile.

In response to this incredible momentum, this year marks the launch of Mobile App World 2010, where global leaders in mobile tech and app development and entrepreneurs will gather to network and learn about the latest developments and innovations.

uTest will be among the outstanding line-up of more than 40 speakers, which includes Google, Microsoft, Ericsson, Orange Global and the BBC, who will be discussing the future of mobile apps. Shoot us a note if you’ll be around!

Note: If you’re looking for some cool, new mobile apps, check out Mobile App World’s August Apps Of The Month. You may spot a uTester’s favorite app!

As the new iPhone 4 hits stores in Israel later this month, more and more attention will be given to the iPhone’s most remarkable feature: APPS! Of course, apps are one of the primary reasons for the latest mobile phone frenzy that has taken the industry by storm. They have revolutionized the cell phone into a portable computing powerhouse. As mobile phones move away from simple communication devices, apps are transforming the mobile phone into a multipurpose tool that can affect every facet of our lives.

So with that in mind, here are the top 5 apps that you must have on your new iPhone 4:

1. Ha’aretz: Stay connected with what’s happening in the world around you! Receive the latest breaking news and headline stories in real time with this app as it happens in Israel and throughout the world.
2. iTranslate: Konichiwa….. Salaam……. Ariverdechi. Translating words from Hebrew is a breeze with this unique app that lets you translate words and sentences from over 50 languages. Impress your friends and loved ones the next time you order something from French restaurant or when your order in Chinese.
3. Aroundme: Find anything from dairy restaurants to gas stations. This app uses GPS technology to find your location and then lists the nearest locations of your search. Your travel experiences will never be the same as everything you could want or need to find using a map or the Internet is now at your fingertips wherever you go.
4. Yad2: From homes to cars to LCD TV’s, you can buy anything your heart desires with this app on Israel’s most famous online 2^nd hand store. This app lets you scroll through the thousands of 2^nd hand items for sale with a simple to use GUI.
5. Israel Money: This app will allow you to make calls from your iPhone in Israel to over 20 countries overseas with absolutely zero charges. There are no charges! No commitments! And no registration!

What’s your favorite iPhone app? How has it helped your daily life? Share with the community which apps are changing the way we use our phones.

(This post was written by Hillel Hurwitz, Community & Content Manager, at Blonde 2.0.)

If you’re in the market for a new mobile device (and these days, who isn’t?) one of the most important decisions you’ll have to make is whether to choose the physical keypad or the touchscreen.

As we learned from our most recent What Do uThink poll, most mobile users actually want both of them. In fact, 70% of respondents said they would prefer a device like the Motorola Droid or the Blackberry Torch, as opposed to the 20% who said they preferred the physical keypad only (e.g. Blackberry Curve, Blackberry Bold) and 10% for the virtual touchscreen (e.g. the iPad, iPhone, etc).

So, 70% of mobile users can”t be wrong, can they? Let’s take a look a few of the arguments (from the uTest Forums thread) in favor of each, then decide for yourself.

Physical Keypad
“I go with Physical keypad. The reason is I have experienced problems while using touchscreen keys. I love Physical keypad which allows me to type the message fast and call the number much quicker than a touchscreen. Also, my LG Android had gone for service because the touch screen was malfunctioning. For instance, when I pressed 3, it used to display 6.”

Hybrid
“I prefer a hybrid model as it gives me independence to use the phone as per my convenience. Also, it makes things much easier for me. If I have to chat or SMS I can use the keyboard feature, but if I have to punch in a phone number or username/password I can do it from touchscreen. Plus, different situations demand things differently so a hybrid model is a perfect one for me.”

Read more…

In the third and final installment of our Testing the Limits interview with Ben Simo, we go back in time to the early 90s to find out how and why he entered the testing profession. We also rapid fire some questions on his browser of choice, his hardware preferences, hobbies and more. In case you missed them, here’s part I and part II.

*********

uTest: Let’s go back in time for a second: How did you get into the craft? What was the first application you tested? What was testing like back in the early 90s?

Simo: Providence. It was providence that got me into testing.

I was young, in love, and planning to get married.  I had been doing some part time database development work, but needed a full time job before the wedding.  I submitted letters and resumes to dozens of companies. I was willing to do almost anything that would pay the rent.  I lived in a city where the local job market was dominated by defense contractors. I quickly learned that many of them called nearly everyone who applied for anything in for an interview; so they could learn about people and add them to databases of potential hires for matching to work they did not yet have.  These companies would then present these people to the government as their available workforce when bidding on contracts. This made it frustrating for those of us looking for work. It often wasn’t clear, when going in for an interview, if it was for a real job or for a potential position that might come at some time in the future if that company were to be awarded a government contract.

I interviewed with the company for which my fiancé (now my wife of 19 years) Sophie worked. It appeared to be one of those information gathering interviews without an actual position to fill. I was asked a lot of questions but none seemed related to a specific opening.  At the end of the interview, the interviewer said he’d be calling me.  Time went by without any more contact. Nearly a month later, I got a call asking if I could start the next morning.

Read more…

In part II of our Testing the Limits interview with Ben Simo, we’ll discuss whether you should trust automated testing tools; the proliferation of testers on Twitter; the true meaning of “QA”; how testing evolves differently in each company; the long lost Bach brothers and much more. You can catch up on the conversation by reading part I. We’ll wrap things up tomorrow with part III.

********

uTest: Jon Bach mentioned that changing the meaning of “QA” to Quality Assistance would help outsiders (engineers, executives, et al) better understand the role of this discipline.  Agree or disagree?

Simo: I believe I first heard  “Quality Assistance”  from Cem Kaner.  I agree with Jon. When testers bear the title Quality Assurance, it often implies that they actually assure the quality of other people’s work. Testers are in a position to help assist quality; not assure it. Let’s not assist the setting of unrealistic expectations with inappropriate titles.

uTest: While we’re on the subject, are you anyway related to James and Jon Bach? The resemblance is uncanny.
Simo: I don’t think so. I’m available for adoption if the Bach family is interested.

uTest: You’ve said that you frequently use automated tools, but that you don’t trust them entirely (back to that whole defensive pessimist thing again). What advice do you have for testers and managers wanting to strike a healthy balance? And what’s currently in your arsenal of automated tools?

Simo: My mistrust in tools is based on the fact that tools can’t think for me. Automated checking can only process whatever decision rules someone thought to program when the checks were created. Automation will consistently do what it is programmed to do and consistently not do what it is not explicitly programmed to do. I find test automation to be useful. In fact, there are some things I’d not want to even try to do manually. I do, however, distrust the green bar. When automated checking passes, I ask myself what the automation does not tell me. I also try to keep aware that people who don’t understand what the automation does are likely to assume that it does more than it does.

Tools are much more than test automation. Tools are essential for testing. I don’t want to test without tools. I have some old programming books that promote testing in which a programmer manually executes code, step-by-step, with pencil and paper in order verify that the code works as expected. This is manual testing. This is a testing practice that came from a time when computer time was rare and cost more than people. We’d now laugh at someone proposing testing in this manner.

Read more…

Our Testing the Limits guest this month is Ben Simo. Known as the “Quality Frog” on Twitter, Ben is one of the most insightful and entertaining testers in the business. A proponent of the context-driven school, Ben has more than 19 years of experience testing software and developing testing tools. He currently lives in Colorado with his wife, two children, two dogs, five cats and fourteen – count ‘em – fourteen goldfish. For the full Ben Simo experience, go to his blog.

In part I of our interview, we get his thoughts on the Worst Bug Ever; his testing philosophy; what it means to be a defensive pessimist; testing certifications, the state of the industry and more. Be sure to check tomorrow for part II.

**************

uTest: Your “Is There a Problem Here?” series has been a big hit in the testing community. What’s the absolute worst bug that’s ever been submitted? And what can testers and developers learn from these type of mistakes?

Simo: Many of the bugs on IsThereAProblemHere.com could be argued to not be bugs. The software works or catches and reports an error condition; but in a way that it unnecessarily frustrates users. My hope is that people involved in creating and testing software can learn from these examples. Rather than only look for the obvious technical bugs, we need to be asking ourselves “Is there a problem here?”

We build software for the benefit of people. Software fails when it does something other than solve human problems.  Although not the worst items submitted, two items come to mind.

The first occurred on Christmas Day last year.  Twitter was full of complaints by people who received Sony’s new electronic book Reader device as Christmas gifts. The device worked except that Sony was not prepared for the Christmas Day rush on their servers as people attempted to install software and purchase books.  By not sufficiently preparing for the Christmas rush on their servers, Sony turned joy into frustration for many new customers. As a performance tester, I take this as a warning to seriously consider what events may cause a surge of demand for the systems I test.

The second problem that comes to mind is one I’ve repeatedly encountered with Blogger’s auto-save feature. I like features that help prevent users from losing their data.  While auto-save features usually indicate that software designers value their customers’ data, Blogger provides a great example of how auto-save can make things worse.  The Ctrl-Z undo option in users’ web browsers goes away after an auto-save occurs.  If a user fat-fingers text in a way that deletes content just before an auto-save occurs, there is no going back. An accidental Ctrl-A instead of a Ctrl-Z or Ctrl-X followed by another keystroke can permanently delete a document in an instant.

uTest: Gotta ask about the “Quality Frog” handle on Twitter. What’s the origin of this moniker?

Simo: A few people have told me “Quality Frog” looks like two random words from a Facebook captcha.

Read more…

Our guest blogger this month is Atul Angra. A resident of India, Atul is one of our more accomplished testers (a Gold Tester in fact), with over six years of professional experience. He’s a photographer at heart, but a tester by trade, with domain expertise in healthcare and finance. He’s also a former Bug Battle winner, a guest judge, a Tester of the Year, a Forums junkie, a crash course author and he’s here today to discuss how interpersonal skills can make or break a tester’s career. Enjoy!

*******

Let’s take a scenario where a tester follows the rules and reports 100 bugs. Some of these bugs were traced to non-documented requirements that are implicit in nature, such as a drop-down list not populating alphabetically and things of that nature. These bugs are quite common and usually end up in conflict, as development teams reject them based on the argument that it’s not a defined requirement.

Here, both the developer and tester are not ready to close this issue – and they are both correct. The traditional way these issues are resolved is by involving someone from management to intervene and make a decision. The time spent in escalation and argument is much greater than what it would have taken to actually fix the issue.

At a high level, we could blame the team which collected requirement, but this may not be the case when it comes to implicit requirements. Many of these situations could be resolved if the tester demonstrates interpersonal skills.

Read more…

In the software testing business, GUI bugs (graphic user interface) are looked down upon – and rightly so. Nobody wants their users navigating through a visually defective application. Software should be clean, polished and aesthetically pleasing. You want your software to be George Clooney, not George Costanza.

But not everyone puts such a high premium on looks. Modern artists, in fact, are doing just the opposite.

Take “glitch art” for example. Defined by Wikipedia as “the aestheticization of digital or analog errors, such as artifacts and other ‘bugs’, by either corrupting digital code/data or by physically manipulating electronic devices,” the trend is attracting more and more mainstream attention.

Wired’s UK site recently posted a fascinating piece on glitch art. They teach us, among other things, that the term is somewhat in dispute. For some, the glitches must be genuine (i.e. unintentional) while others insist that glitch art can be deliberately created, mainly through a technique known as data bending:

Read more…

Rumors of the mobile web’s death have been greatly exaggerated. Despite some compelling arguments in Wired’s latest series -  where experts assert that native apps have (or will soon) totally displace the web as a medium of choice – we’re not quite ready to pull the plug. Apparently, neither is the general public. Not just yet.

More on that in a second, but first, let’s examine why some are making this claim. It’s true that there’s been a meaningful shift towards native apps over the last few years, thanks mostly to the iPhone and its offspring (i.e. smartphones). What was once the Great Wide Open, the Internet has been parceled into what Wired calls “semiclosed platforms that use the Internet for transport but not the browser for display.”

In other words:

You wake up and check your email on your bedside iPad — that’s one app. During breakfast you browse Facebook, Twitter, and The New York Times — three more apps. On the way to the office, you listen to a podcast on your smartphone. Another app. At work, you scroll through RSS feeds in a reader and have Skype and IM conversations. More apps. At the end of the day, you come home, make dinner while listening to Pandora, play some games on Xbox Live, and watch a movie on Netflix’s streaming service….

You’ve spent the day on the Internet — but not on the Web. And you are not alone…

Quite true. But you are also NOT alone if you’re still using the mobile web. As part of our weekly “What Do uThink” poll question, we asked our community whether they prefer to get information via native apps or the mobile web. Here were the results:

Read more…