Software Testing Blog

Our first-ever “Bug of the Month” competition came to a close on August 29 – and what a month it was! More than 4,000 – that’s right 4,000 – people joined uTest on Facebook to partake in the “testivities.” And just as we’d hoped, we received a lot of interesting bugs, error messages and easter eggs over the last few weeks — ranging from weird to wacky to scary, and everything in between. It all went according to plan. Excellent.

Who knew that this kind of social contest would get people so fired up (not to mention so competitive)? The contest really heated up in the final weeks, as the number of “likes” climbed higher and higher with each passing day. And similar to politicians in a “too close to call” election, participants hit the campaign trail hard in the final days, trying to convince the app testing world that their entry deserved to win.

As a result of  this overwhelming response to our inaugural contest, we decided to up the ante and add two extra prizes into the mix. So, after 4, 406 “likes” and a collection of captured bugs, we’re excited to announce the winners:

• In 1st place was Abagail Gm from the Philipines with 1844 “likes.” She walks way with a new iPod Touch for her efforts.
• Manas Mehta from India takes the 2nd prize, a trendy iPod Nano for the bug with the second highest number of “likes.”
• And the 3rd prize goes to Amos Naor for our Judge’s Choice Award (the most interesting and thought-provoking easter egg– and one we had never seen before)  For unearthing this gem, Amos will receive a stylish iPod shuffle.

Plans are already underway for the next app-related contest, so be sure to connect with us on the uTest Facebook page. In the meantime, keep your eyes pealed for the next Bug of the Month.

(The warped & witty writers over at Blonde 2.0 contributed to this post)

Apps! Apps! And more apps! As the summer starts winding down here at uTest, we’ve been able to take a step back and a closer look at the big trends emerging all around us. What has been most apparent is the tremendous spike in mobile app testing needs. From top marketing agencies to retail giants to social gaming startups, our customers are developing more mobile apps to grow (or define) their businesses than ever before.

According to Game Developer Research, 25% of game developers are now making mobile games – that’s up from a mere 12% in 2009!

In addition, a survey conducted by iGR found that more than half (53%) of US mobile developers are building apps for Apple’s iPhone OS. BlackBerry was the next most popular, followed by Android and Windows Mobile.

In response to this incredible momentum, this year marks the launch of Mobile App World 2010, where global leaders in mobile tech and app development and entrepreneurs will gather to network and learn about the latest developments and innovations.

uTest will be among the outstanding line-up of more than 40 speakers, which includes Google, Microsoft, Ericsson, Orange Global and the BBC, who will be discussing the future of mobile apps. Shoot us a note if you’ll be around!

Note: If you’re looking for some cool, new mobile apps, check out Mobile App World’s August Apps Of The Month. You may spot a uTester’s favorite app!

As the new iPhone 4 hits stores in Israel later this month, more and more attention will be given to the iPhone’s most remarkable feature: APPS! Of course, apps are one of the primary reasons for the latest mobile phone frenzy that has taken the industry by storm. They have revolutionized the cell phone into a portable computing powerhouse. As mobile phones move away from simple communication devices, apps are transforming the mobile phone into a multipurpose tool that can affect every facet of our lives.

So with that in mind, here are the top 5 apps that you must have on your new iPhone 4:

1. Ha’aretz: Stay connected with what’s happening in the world around you! Receive the latest breaking news and headline stories in real time with this app as it happens in Israel and throughout the world.
2. iTranslate: Konichiwa….. Salaam……. Ariverdechi. Translating words from Hebrew is a breeze with this unique app that lets you translate words and sentences from over 50 languages. Impress your friends and loved ones the next time you order something from French restaurant or when your order in Chinese.
3. Aroundme: Find anything from dairy restaurants to gas stations. This app uses GPS technology to find your location and then lists the nearest locations of your search. Your travel experiences will never be the same as everything you could want or need to find using a map or the Internet is now at your fingertips wherever you go.
4. Yad2: From homes to cars to LCD TV’s, you can buy anything your heart desires with this app on Israel’s most famous online 2^nd hand store. This app lets you scroll through the thousands of 2^nd hand items for sale with a simple to use GUI.
5. Israel Money: This app will allow you to make calls from your iPhone in Israel to over 20 countries overseas with absolutely zero charges. There are no charges! No commitments! And no registration!

What’s your favorite iPhone app? How has it helped your daily life? Share with the community which apps are changing the way we use our phones.

(This post was written by Hillel Hurwitz, Community & Content Manager, at Blonde 2.0.)

A couple weeks ago, we rolled out v3.0.4 of our platform. And now, after a little extra time in the oven, we’ve launched another feature designed to make it quicker & easier to access your uTest account.

Social Sign-In
Let’s say you’re like millions of other people around the world and you have an account on Facebook, Twitter, or LinkedIn. And let’s also say you’re tired of keeping track of passwords, remembering to sign-in everyday to all the websites you like, and are yearning for something simpler. Starting today, we have a solution for you, at least when it comes to your uTest account (we can’t speak for anyone else). Now you can log in to the uTest platform using the credentials from your Facebook, Twitter or LinkedIn accounts.

Setting up this linkage is simple:

1. Get started by visiting our platform and clicking the button for your favorite social network(s).
2. Then sign in to your social network and confirm the connection.
3. And that’s pretty much it. The next time you visit the uTest platform, just click that same button and you’re in.

Enjoy!

If you’re in the market for a new mobile device (and these days, who isn’t?) one of the most important decisions you’ll have to make is whether to choose the physical keypad or the touchscreen.

As we learned from our most recent What Do uThink poll, most mobile users actually want both of them. In fact, 70% of respondents said they would prefer a device like the Motorola Droid or the Blackberry Torch, as opposed to the 20% who said they preferred the physical keypad only (e.g. Blackberry Curve, Blackberry Bold) and 10% for the virtual touchscreen (e.g. the iPad, iPhone, etc).

So, 70% of mobile users can”t be wrong, can they? Let’s take a look a few of the arguments (from the uTest Forums thread) in favor of each, then decide for yourself.

Physical Keypad
“I go with Physical keypad. The reason is I have experienced problems while using touchscreen keys. I love Physical keypad which allows me to type the message fast and call the number much quicker than a touchscreen. Also, my LG Android had gone for service because the touch screen was malfunctioning. For instance, when I pressed 3, it used to display 6.”

Hybrid
“I prefer a hybrid model as it gives me independence to use the phone as per my convenience. Also, it makes things much easier for me. If I have to chat or SMS I can use the keyboard feature, but if I have to punch in a phone number or username/password I can do it from touchscreen. Plus, different situations demand things differently so a hybrid model is a perfect one for me.”

Read more…

Pardon my American “gun-loving” nature while I start this blog post with an analogy about guns. Let’s say that I’m interested in learning how to shoot a gun, so I go visit my local shooting range for instruction. The people who run the shooting range (which is fully licensed and very reputable) give me extensive safety training before finally issuing me a gun with some ammunition. While standing in front of the target, I carefully load the gun, point the barrel at my foot, and pull the trigger.

So whose fault is it that I just shot myself in the foot? Is it the shooting range’s for not issuing me special bullet proof boots or for not training me even more thoroughly about the dangers of shooting my extremities? Or is it mine, for shooting myself in the foot?

A recent Windows “bug” has put Microsoft in the position of the shooting range. I put “bug” in quotes because it’s hardly the shooting range’s fault that I shot myself in the foot, but it’s still their problem in the end. Same with Microsoft who must now deal with an annoying flaw in their DLL loading behavior that isn’t their fault but has become their problem. Ars Technica explains the problem very well, but here’s a quick summary of the issue:

Read more…

In the third and final installment of our Testing the Limits interview with Ben Simo, we go back in time to the early 90s to find out how and why he entered the testing profession. We also rapid fire some questions on his browser of choice, his hardware preferences, hobbies and more. In case you missed them, here’s part I and part II.

*********

uTest: Let’s go back in time for a second: How did you get into the craft? What was the first application you tested? What was testing like back in the early 90s?

Simo: Providence. It was providence that got me into testing.

I was young, in love, and planning to get married.  I had been doing some part time database development work, but needed a full time job before the wedding.  I submitted letters and resumes to dozens of companies. I was willing to do almost anything that would pay the rent.  I lived in a city where the local job market was dominated by defense contractors. I quickly learned that many of them called nearly everyone who applied for anything in for an interview; so they could learn about people and add them to databases of potential hires for matching to work they did not yet have.  These companies would then present these people to the government as their available workforce when bidding on contracts. This made it frustrating for those of us looking for work. It often wasn’t clear, when going in for an interview, if it was for a real job or for a potential position that might come at some time in the future if that company were to be awarded a government contract.

I interviewed with the company for which my fiancé (now my wife of 19 years) Sophie worked. It appeared to be one of those information gathering interviews without an actual position to fill. I was asked a lot of questions but none seemed related to a specific opening.  At the end of the interview, the interviewer said he’d be calling me.  Time went by without any more contact. Nearly a month later, I got a call asking if I could start the next morning.

Read more…

In part II of our Testing the Limits interview with Ben Simo, we’ll discuss whether you should trust automated testing tools; the proliferation of testers on Twitter; the true meaning of “QA”; how testing evolves differently in each company; the long lost Bach brothers and much more. You can catch up on the conversation by reading part I. We’ll wrap things up tomorrow with part III.

********

uTest: Jon Bach mentioned that changing the meaning of “QA” to Quality Assistance would help outsiders (engineers, executives, et al) better understand the role of this discipline.  Agree or disagree?

Simo: I believe I first heard  “Quality Assistance”  from Cem Kaner.  I agree with Jon. When testers bear the title Quality Assurance, it often implies that they actually assure the quality of other people’s work. Testers are in a position to help assist quality; not assure it. Let’s not assist the setting of unrealistic expectations with inappropriate titles.

uTest: While we’re on the subject, are you anyway related to James and Jon Bach? The resemblance is uncanny.
Simo: I don’t think so. I’m available for adoption if the Bach family is interested.

uTest: You’ve said that you frequently use automated tools, but that you don’t trust them entirely (back to that whole defensive pessimist thing again). What advice do you have for testers and managers wanting to strike a healthy balance? And what’s currently in your arsenal of automated tools?

Simo: My mistrust in tools is based on the fact that tools can’t think for me. Automated checking can only process whatever decision rules someone thought to program when the checks were created. Automation will consistently do what it is programmed to do and consistently not do what it is not explicitly programmed to do. I find test automation to be useful. In fact, there are some things I’d not want to even try to do manually. I do, however, distrust the green bar. When automated checking passes, I ask myself what the automation does not tell me. I also try to keep aware that people who don’t understand what the automation does are likely to assume that it does more than it does.

Tools are much more than test automation. Tools are essential for testing. I don’t want to test without tools. I have some old programming books that promote testing in which a programmer manually executes code, step-by-step, with pencil and paper in order verify that the code works as expected. This is manual testing. This is a testing practice that came from a time when computer time was rare and cost more than people. We’d now laugh at someone proposing testing in this manner.

Read more…

Our Testing the Limits guest this month is Ben Simo. Known as the “Quality Frog” on Twitter, Ben is one of the most insightful and entertaining testers in the business. A proponent of the context-driven school, Ben has more than 19 years of experience testing software and developing testing tools. He currently lives in Colorado with his wife, two children, two dogs, five cats and fourteen – count ‘em – fourteen goldfish. For the full Ben Simo experience, go to his blog.

In part I of our interview, we get his thoughts on the Worst Bug Ever; his testing philosophy; what it means to be a defensive pessimist; testing certifications, the state of the industry and more. Be sure to check tomorrow for part II.

**************

uTest: Your “Is There a Problem Here?” series has been a big hit in the testing community. What’s the absolute worst bug that’s ever been submitted? And what can testers and developers learn from these type of mistakes?

Simo: Many of the bugs on IsThereAProblemHere.com could be argued to not be bugs. The software works or catches and reports an error condition; but in a way that it unnecessarily frustrates users. My hope is that people involved in creating and testing software can learn from these examples. Rather than only look for the obvious technical bugs, we need to be asking ourselves “Is there a problem here?”

We build software for the benefit of people. Software fails when it does something other than solve human problems.  Although not the worst items submitted, two items come to mind.

The first occurred on Christmas Day last year.  Twitter was full of complaints by people who received Sony’s new electronic book Reader device as Christmas gifts. The device worked except that Sony was not prepared for the Christmas Day rush on their servers as people attempted to install software and purchase books.  By not sufficiently preparing for the Christmas rush on their servers, Sony turned joy into frustration for many new customers. As a performance tester, I take this as a warning to seriously consider what events may cause a surge of demand for the systems I test.

The second problem that comes to mind is one I’ve repeatedly encountered with Blogger’s auto-save feature. I like features that help prevent users from losing their data.  While auto-save features usually indicate that software designers value their customers’ data, Blogger provides a great example of how auto-save can make things worse.  The Ctrl-Z undo option in users’ web browsers goes away after an auto-save occurs.  If a user fat-fingers text in a way that deletes content just before an auto-save occurs, there is no going back. An accidental Ctrl-A instead of a Ctrl-Z or Ctrl-X followed by another keystroke can permanently delete a document in an instant.

uTest: Gotta ask about the “Quality Frog” handle on Twitter. What’s the origin of this moniker?

Simo: A few people have told me “Quality Frog” looks like two random words from a Facebook captcha.

Read more…

While there’s a lot of debate about what constitutes good testing and acceptable levels of quality assurance, can we all agree that your app should not cripple your testers and users?  Ok, good.

And if this does resemble your launch decisions a little too closely (seriously, I hope not), give us a call.  Happy weekend, everyone!