Software Testing Blog

As the smartphone battle heats up, a debate has begun around a seemingly crucial question: which platform is faster?  In a lot of ways, that’s impossible to answer. Performance comparisons depend on many factors, including the tradeoff between performance and battery life.  But that hasn’t stopped some from having the debate anyway, and the battle lines right now are over Android’s JVM vs. the iPhone’s Objective C objc_msgSend().  Let me explain.

Android is a Java based platform and uses a Java Virtual Machine or JVM to execute apps (Android’s JVM is called Dalvik). Historically, Java was considered to be a slow and cumbersome platform.  The joke was write once, run anywhere very, very slowly.

So people are saying that Android is slow, right?  Actually, no.  In fact, Java has been optimized so extensively in the past 10 years that its performance is now incredibly fast.  Dalvik has been optimized even further for mobile devices, so Android is one fast platform. What people are actually saying is that the iPhone is slow.

Read more…

In this month’s installment of This Week In Testing, the date was May 1996 and the setting was the First National Bank of Chicago (insert dramatic pause here). The gist? Software “glitches” caused the bank accounts of 823 customers of the major US bank to be credited with a total of $924,844,208.32 each.

According to The American Bankers Association, all of $763.9 billion — more than six times the total assets of First Chicago NBD Corp. — was the largest error in US banking history.

And the reason given? Inadequate testing of course! The bank updated its ATM transaction software with new message codes. The message codes were unfortunately not tested on all ATM protocols, which resulted in some ATMs interpreting the codes as huge increases to customer balances.

This isn’t the first time we bring up banking bugs. You might remember Software Bugs: You Win Sum, You Lose Sum, the post about a man in Orlando who while making a routine bank transfer was shocked to see his balance at $88,888,888,888.88.

What other bugs have you recently heard or read about with such huge financial implications? Any mobile banking bugs?

Back in December, we interviewed the notable, quotable James Bach as part of our Testing the Limits series. At the time, James had just published Secrets of a Buccaneer-Scholar, where he advocates seeking out alternatives to traditional schooling. For software testers, this meant looking beyond industry certifications.

About a month ago, James was again interviewed on the subject of “buccaneer testing”, except this time it was caught on camera by Yvette Francino of TechTarget (while covering StarEast 2010).

Check out this inspiring, two-minute explanation of buccaneer testing:

Happy buccaneer testing, mateys!

Two weeks ago the game company Valve launched Steam, their online game distribution platform, for the Macintosh.  Mac gamers rejoiced at the availability of a wide library of video games, and then they rejoiced even more when Valve announced that Portal, their award-winning game and cult classic, would also be available for free on the Mac and PC.

Of course a game like Portal hit uTest hard, sweeping its way through the offices like a bad cold in winter.  One by one we each became addicts.  But as games go Portal is short, and we all quickly returned to our software testing ways not long after.  So besides us all letting off some steam (no pun intended), what does this have to do with software testing?  Well, it turns out that the fine folks at Valve have been watching Mac vs. PC performance using monitoring tools built-in to Steam, and their results are quite surprising.

Read more…

Last week, we posted a pretty powerful piece about the loyalty of crowds for all those crowdsourcing veterans out there — both skeptics and champions alike — to start demystifying the “human nature” of online communities. We received some really great feedback (thank you!), and realized that many people still haven’t heard the basics of the crowdsourcing model. For those of you who just need the quick nuts and bolts, here’s a pretty good interview on CBS NEWS that may be useful: What Is Crowdsourcing?

Watch CBS News Videos Online

Worldwide productivity surely took a nose-dive today, as thousands of worker bees (like me) discovered that Google was featuring the classic Pac-Man arcade game on its ever-changing homepage. What began as a scholarly search for “regression testing tips” quickly devolved into “five” minutes of ghost-chomping fun – but don’t tell my boss.

In honor of Pac-Man’s 30th birthday, Google developed the application (in what we presume is HTML5) to look, sound and behave just like the original version from 1980. [UPDATE: Here's how to download the Pac-Man game for free] We’re not yet sure if this includes the infamous Pac-Man kill-screen bug, but I am determined to find out. I’ll work nights and weekends if that’s what it takes. That’s just the kind of dedicated employee I am.

Anyway, since we’re a software testing company, many of us in the office were curious to see how the application would perform on the various mobile devices we have in-house. Here’s a quick run-down of our findings for each device, including whether or not it worked, along with a few notes:

Read more…

Depending upon who you ask, crowdsourcing is either evil, revolutionary, or a next gen of internships.

But one thing that ALL crowdsourcing companies like to preach is how loyal and trustworthy and professional their community is. I know because I’ve read it in 100 different sites. Hell, I’ve written it a 100 different times here at uTest. So why do crowdsourcing companies insist upon telling the world how loyal and earnest their community is?  Maybe it’s to assuage the fears of prospective customers about entrusting their logo design, app development, content production or marketing to a community of strangers. Maybe it’s because if marketers say it enough times, we hope it’ll come true.

The more pessimistic view is that people — cloaked in the anonymity of the web — often act in greedy, selfish, mean-spirited ways (this perspective didn’t make it into the crowdsourcing brochure, by the way). Such dark behavior is well-documented and takes the form of flame wars on message boards, bullying via social media and online fraud.

So which is it — are people good-natured and honest?  Or are they money-hungry malcontents who will do anything to get ahead, as long as they don’t get caught?  Obviously it depends on the people, but I learned the truth about our community this week — and it was a lesson we learned the hard way.

A little background:  At uTest, we pay our testers twice per month via PayPal or Payoneer. And at this point in our growth, each pay cycle involves a non-trivial amount of cash — pretty deep into the five-figure range.  Now, it’s not easy or flattering to admit this, but in our most recent pay cycle, we experienced a glitch that caused us to pay our testers twice. That’s right folks, it was double payday here at uTest!

Read more…

We’re about half way through the Q2 Bug Battle, which is comparing three of the most popular “check-in” applications. That would be Foursquare, Gowalla and Brightkite.

So how’s it going so far? Well, with less than five days remaining, we’ve already had more than 350 testers participate and a combined total of  650+ reported bugs!  The depth and quality of the defects is nothing short of amazing, but we’ve come to expect no less from our tester community.

Of course, there’s still a long way to go – and nearly $4,000 in prize money still up for grabs. Remember, the Bug Battle ends Monday, May 24th at noon ET, so be sure to log in to your uTest account and start submitting bugs before it’s too late.

As with previous competitions, it’s all about quality and not quantity. If you want more specific tips on how to win the Bug Battle – by focusing on mobile bugs, social media integration, etc. – then check out this dedicated thread in the uTest Forums.

Happy bug-hunting and good luck!

In the third and final installment of our Testing the Limits interview with Lanette Creamer, we cover the Seattle testing scene; why more women don’t enter the profession; mobile testing challenges; test automation; her favorite Nicholas Cage movie and more. In case you missed them, here’s part I and part II.

uTest: The Seattle area has spawned an inordinate number of top testers (Whittaker, Bach, Bach, Creamer, et al) – what’s the deal with that?  Is there something in the water or is just a result of the Microsoft ecosystem being nearby?
LC: If there was no James Bach there would be no interview with a crazy redheaded tester named Creamer, because I would have no testing blog. If James Bach wasn’t in Seattle, I may not have had the chance to see him speak so often. Cast 2007 was in Bellevue, WA, maybe partially because that is close to Microsoft, so I guess in a roundabout way, it could be the Microsoft ecosystem being nearby that made Bellevue the location for Cast at the right time.

I prefer to think of it as something special about Seattle that fosters a unique perspective and resilience. Maybe it’s all of the cloudy weather. The grunge movement started in Seattle, and much like Nirvana, Pearl Jam, and Soundgarden, there are some innovative contrarians who aren’t afraid to blaze new trails coming out of Seattle to this day – and flannel is in style again.

uTest: Numbers-wise, the software testing profession is clearly dominated by dudes. Why do you think that is? How do we change this trend? Does it matter, or is this topic completely overblown?
LC: When all of the women who have the talent, skills, and desire to be testing are appreciated for the value they offer, and the field is still dominated by dudes, then great! It is about having the opportunity, not about enforcing some gender ratio. Right now things are not equal and fair for female testers and I’d like to see that change in my lifetime. I don’t think male testers are the problem at all. After a few curious looks, once we start actually testing or talking about it, in my experience, most testers are supportive and eager to help each other learn regardless of gender. The problem is higher up in the companies where the value testers bring isn’t well understood and diversity isn’t valued for men or women. The top reason we should care about diversity in our testing teams is because the demographic of a computer user is more diverse than ever before.

Read more…

In part II of our Testing the Limits interview with Lanette Creamer – aka “Testy Redhead” – we cover the need for Exploratory Testing; Matt Heusser and the “rebel alliance” of testing; how to create a popular testing blog; her stance on tester certifications and more from the wide world of QA. Catch up by reading part I, and when you’re done with this one, go check out part III.

uTest: In one of your recent blog posts, you mention Elisabeth Hendrickson’s STAREAST declaration that “Exploratory Testing Is Not Optional.” Why did this statement resonate so strongly for you? Do you think all testing managers should follow Hendrickson’s lead?

LC: As a frequent conference attendee and enthusiastic reader of testing blogs, I’ve seen many ideas about how to improve testing. I’ve been through countless industry trends, such as borrowing manufacturing ideas, extensive measuring schemes, and repeated attempts to automate all testing. The bottom line is exploratory testing works in practice. Not for a few months or years, but it works to find important bugs year after year no matter what other quality trends are happening. It works well side by side with automated checks, manufacturing ideas, and it can be used with session based test management to provide measures and metrics if needed. It is the one constant a tester can go back to and find bugs that impact the user experience. It is the meat in my testing sandwich. (My pun filled humor is the cheese.) To hear Elisabeth acknowledge the importance of exploratory testing in public shows me that agile testing is about more than just automation. Agile testing can be about a balanced approach to overall quality. It resonated with me strongly because it makes me hopeful for the future of testing on agile teams.

I think test managers should evangelize and defend what works well in practice on their teams. My experience has been that exploratory testing is generally undervalued considering how effective and practical it is.

uTest: What’s the deal with this “rebel alliance” thing we’ve been hearing so much about? It sounds subversive – we want in! Seriously though, what’s the mission of this group? Please explain it to our un-initiated readers.

Read more…