Google: Your RAM is Out to Get You!

Uh-oh! Better report that bug!

Your computer just crashed taking with it that document you’ve been working on for the past hour.  Naturally, you cry out in anguish over all those terrible software bugs that conspire to crash your computer and drive you nuts. But new evidence suggests that something else could be the cause: bad memory.

Google just announced the results of a new study based on the thousands of servers they manage in their data centers, and the results are surprising.  Memory error rates are thousands of times higher than anyone previously believed.  (CNET has a good summary of the research here.)

For software testers, this represents a significant new wrinkle in identifying bugs.  Wonky behavior and computer crashes could really be a hardware problem, and these sorts of hardware issues are way more common than we previously thought.

So what does that mean for your testing?  Here are a few thoughts that could help.

Continue Reading

A Bug of a Different Kind

Gabe and Tycho of Penny Arcade are normally hilarious.  Their thrice weekly comic is a must read for anyone who loves gaming, and their yearly logo PAX conference has grown to be one of the most influential gamer gatherings in the world.

While games are prone to having bugs, this year’s conference (which ended Sunday) also featured a bug of a much scarier kind: H1N1 flu.  Over one hundred gamers have been diagnosed with the virus, many of whom have since traveled back home carrying it with them.

So as a reminder – get to know the symptoms of H1N1 flu and see a doctor if you feel ill.  Stay healthy everyone!

The Boston Test Party

Last night, uTest co-hosted our second Boston-area tester meetup with our UK pest_2realpartner, TCL, at The Kinsale. About 25 of local software testers showed up ready for some testing games, pizza, beer and networking fun at this P.E.S.T. (Pub Exploration Software Testing) event.

pest_1Attendees got a sneak peek at a soon-to-be-released version of the uTest platform and went at it to show off their web testing skills. Those who discovered the most creative and valuable bugs were awarded an iPod Touch, an iPod Nano and an iPod Shuffle.

We had a great time networking with Boston’s testers, developers and students.  We hope to see you soon in the uTest community reporting showstopper bugs for the biggest software companies in the world.

Great to meet you all in person and we look forward to doing it again later this year.  Stay tuned for details on our next P.E.S.T. event.

Check out the uTest Flickr page to see more pics from this event and other uTest gatherings.

Name That Plague (software testing plague, that is)

Frequent readers of the uTest blog are by now aware that we’re big fans of James Whittaker – software testing expert,james_whittaker author and now one of Google’s top QA guys. Over the past few weeks, James has been writing a provocative series called the “Seven Plagues of Software Testing”. You can find it on his new blog.

As you’ll notice, only six of the plagues have been published thus far. The seventh and final software testing plague was intentionally omitted, as he is accepting submissions from his readers.

Before you send him your suggestions (we included an email address at the bottom of this post), here’s a few excerpts from some of the plagues he’s discussed so far:

The Plague of Blindness: “Software testing is much like game playing while blindfolded. We can’t see bugs; we can’t see coverage; we can’t see code changes. This information, so valuable to us as testers, is hidden in useless static reports. If someone outfitted us with an actual blindfold, we might not even notice.”

Continue Reading

Bug Battle Part III: Twitter Apps!

After web browsers and social media sites, you were probably wondering how we’d top ourselves for the next uTest Bug Battle. Well, after months of debate and deliberation, uTesters will compete to find bugs in six of the top Twitter desktop apps!

Unless you’ve been asleep, in a coma, or camping in the wilderness for the past two years, you’re likely aware that the Twitterverse is expanding at a mind boggling rate (thanks, Ashton and Oprah). And since hundreds of apps have been created around the popular micro-blogging site, we figured they’d make a perfect subject for our now famous Bug Battle. Here are the Twitter apps we’ll be testing, in no particular order:

1.    Tweet Deck 0.25
2.    Seesmic DESKTOP
3.    Twhirl 0.9.2
4.    Tweetr 3.4
5.    Twitterific 3.2 (mac only)
6.    Twitteroo 1.5 (pc only)

The contest kicks off RIGHT NOW (12:01am ET on Thursday) and will run through 11:59pm ET next Wednesday, June 3rd.

During that time, members of our QA community will be searching for the most serious, compelling bugs they can find, including technical, functional and GUI bugs. uTesters can focus their efforts on as many or as few of these as they choose.  We will be awarding more than $3,000 in prize money for:

* Top overall tester (based on quality of bugs and feedback)
* Top novice tester (same criteria)
* Top US-based tester (same)
* Top five individual bugs (severity and complexity)
* Best feedback (post-contest survey about the feature set, functionality and usability)

You can get more info on the Twitter Bug Battle. To find out how you – yes you! – can win the money finding bugs in these Twitter applications, be sure to check out the Bug Battle thread in our testers-only forums.

Have questions or ideas for future Bug Battles? Drop us a comment. Want to keep up with the Bug Battle action while it’s going on?  We’ll be sharing real-time updates throughout the weeklong Twitter contest on (where else?) Twitter.

uTest Relaunch in the News!

The word is out!  With the launch of our revamped website, a new “Meet the Testers” application and Tester Forums, last week uTest made significant strides in turning our crowd of 16,000+ QA professionals into an interactive community.

Here’s what the media had to say:

  • crowdsourcingformatted1Jeff Howe, author of best-selling book Crowdsourcing & contributing editor at Wired Magazine, twittered the news to his 2,000 followers calling us out as a Company to Watch.

Other notable posts at  StickyMinds.com, the Cloud Architect, and MSNBC.com.

Let us know what you think about the new site!

uTest Hitting The Road This Week

This evening, Doron Reuveni will be joining a new meetup here in Boston called the Ultra Light Startups. The meetup is a resource for entrepreneurs to share best practices for launching tech startups. The topic is near andultralight dear to our hearts: Crowdsourcing. If you’re in the neighborhood, come join us at Boston University at 595 Comm. Ave. Doors open at 6pm! Questions the panelists (Local Motors, Acquia, GeniusRocket and uTest) will tackle:

  • What is crowdsourcing?
  • What are the benefits?
  • Why is it a disruptive model?
  • What are the most effective ways to build communities?

Doron will also be flying down to Orlando to present at STAREAST 2009 on Thursday morning. His presentation will introduce this new era of community-based software testing and delve into how companies can launch higher quality apps while staying within budget through crowdsourcing. Challenges like shorter release cycles, increased customer expectations, smaller budgets and fewer testing resources are forcing us to rethink our stareastQA methods.

Doron will discuss how crowdsourced testing helps to meet these challenges head on. Here’s your chance to take the kids to Disney and be a part of the largest and most advanced testing forum to keep you up on the latest trends, technologies, and strategies in the industry today. Shoot us a note if you’ll be down there!

Testing Tip: Screenshots on Firefox

Without a doubt, one of the most important things you can submit with a bug is a screenshot.  For those of you using Firefox, check out the excellent Screengrab extension.  It’s completely free and allows you to take a screenshot of the entire page at once rather than just what’s visible in your window.  It also saves the files in PNG or JPG formats – making them smaller and more manageable than TIFF or BMP files.

Take a look at this screenshot of the homepage for the Software Testing Club:

software-testing-club-online-qa-software-testing-community_1239721283757

Only half of that would be visible in an ordinary browser window.  The plugin also lets you capture selections of a page, so you can also use it to highlight important errors.

Here’s a bonus link.  Check out this excellent post on Mashable with 7 Useful Tools for Web Development Testing.

Bug Battle Builds Big Buzz

Apologies for the alliteration, but things have been ultra-busy around the halls of uTest this week.  The reason our phones have been ringing off the hook is because we announced the results of our 1st Quarter Bug Battle on Tuesday.

In case you missed it, we’ve received some fantastic media coverage in the past few days.  Who knew the world would be so interested in the results of 1,000+ QA professionals from 64 countries simultaneously testing the world’s three most popular networking sites?  Ok, well we had a pretty good idea they’d care, but we’re glad it captured their attention.  Here are a few of the noteworthy articles:

- TechCrunch: uTest Bug Battle: Which Social Network Is The Buggiest?
- Dr. Dobbs: Bugs In Social Networking Software? You’re Kidding, Right?
- eWeekFacebook Triumphs LinkedIn, MySpace in uTest’s Bug Battle

Our community’s ability to mobilize quickly and provide real-world testing coverage is making a strong impression on a growing number of companies.  These firms are looking for ways to do more with less — and the uTest community is primed to help them meet their QA needs.  Keep up the good work!

How Many Bugs Do You See?

Like many Americans, I’ve been watching housing prices very intently.  While looking at data on the website for the Massachusetts Association of Realtors, I came across a very common and ordinary web bug: a broken link.

Screenshot

(Click the thumbnail for a full sized version.)

Now broken links are incredibly common, and anyone with a website has had one happen to them at some point.  But I want to dig a little deeper.  Take a look at that screen shot and think about how many bugs do you see?

By my count, I see three very real bugs:

  • There’s the broken link.  Looking at this error, I think this is a convoluted way of saying that the file I wanted wasn’t found.  Whoever uploaded the file didn’t set a folder correctly.
  • The Folder is not set exception didn’t get handled.  Ideally, some kind of exception handler should have caught this exception and provided the user with a more coherent error message.
  • The user was shown an incoherent error message.  An end user should never see this kind of error message in the first place.  It reveals information about the underlying system configuration that could be dangerous.  At the same time the user doesn’t receive any kind of friendly help about why the error occurred or how to resolve it.

Each of these is a very real bug, but each one is also more difficult to fix than the previous.  For example, fixing a broken link may be as simple as changing a file location in the content management system.  The person responsible for posting the content could take care of that.

Correcting the next bug requires help from the developers to write a proper exception handler to catch this exception. While fixing a broken link is something that could be handled in the field, writing a new exception handler usually has to be done during the development process.  This, of course, requires a solid QA process to be in place during development.

For most projects, the last bug is the most challenging to fix.  It usually requires careful planning from the earliest stages by product management, development, and QA to agree that reducing the complexity of bugs is both important and feasible.  Software testers will have to be involved early in the process so that they can understand how to discover and interpret bugs that are designed to be customer friendly or even invisible.

Even simple bugs, like broken links, can lead to deeper questions for testing and development teams.  Not every bug has to be fixed, but make sure to keep your customers and users in mind when planning your project.  Good planning could turn a boring broken link into something that’s both apologetic and helpful.