Software Testing Blog

We really can’t believe an entire year has gone by (BTW: what a fantastic ride it’s been!), but yesterday the Boston Business Journal included us in its top list of Best Places To Work (only 25 in small co. category) for the second year in a row!

We are so honored to be recognized in back-to-back years and listed next to amazing companies that demonstrate dedication to their employees’ professional growth and career development and provide a ROCKIN’ work environment.

The winners ranked highest among hundreds of companies whose employees filled out an anonymous survey with detailed questions regarding their overall work environment and experience.

The list includes global enterprises such as Google, Microsoft, Ritz-Carlton, and Comcast, as well as local startups such as Carbonite and HubSpot.

As Doron said, “We’re committed to achieving great commercial success, but we also strive to create a rewarding and meaningful experience for our 40+ in-house employees. They are the ones who make uTest’s success possible!”

THANK YOU to the entire uTest crew who make uTest not only a best place to work, but simply the best place to be day-to-day. There’s no way we would be where we are without our world-class team.

Read more…

Sometimes, we interview a testing expert with so much wisdom, so much experience and so much to say, that one appearance simply won’t do. Jon Bach is one of those experts – and he’s back for his second Testing the Limits interview (Bach to the Future…anyone?). Jon recently joined eBay as the new QA Director for their Buyer Experience department. Prior to his new role, he had stints at Quardev Laboratories, Microsoft, Rational, LexisNexis, and Hewlett-Packard. Impressive track record, eh?

In part I of the interview, Jon discusses his new role at eBay; knowledge transfer in testing; war stories in software testing; how to test with no requirements and more. When you’re done here, go read Part II.

*****

uTest: First off, congrats on your new job as the QA Director for eBay. I know you’re only a few weeks in, but what are you most looking forward to in this new role? And how is the transition going so far?

JB: I work for an aptly-named eBay department: “Buyer Experience: Search and Discovery”.  My team handles anything that has to do with what we call “Search Front End” – the UI elements rendered and functions that support searches for merchandise on eBay. eBay does more searches per day than Google, and I’m looking forward to finding search issues that affect millions of people every day.

As for “transition”, I have my laptop, badge, and a name tag for my office; bought myself an eBay hat, but the most important “transition” is one of mindset.  I used to think eBay was kind of like a pawn shop or a swap meet, but it’s got millions of new items from major manufacturers.  It’s a platform for worldwide ecommerce, whether it’s items sold by individuals or major retail outlets.

The most important transition for me is changing my own ecommerce habits.  Instead of going to Amazon to buy books, for example, it’s more useful for me as a tester and as an eBay employee to think “can I get it from eBay first?” Recently, I needed to order Valentine’s Day flowers for my wife.  I started going to proflowers.com when I thought, “can I get them through eBay?” Turns out that eBay options for flowers are not displayed very prominently on the site.  I finally found text ads for florists on the bottom of one of eBay’s search results pages, but I had to dig to find them.  This caused me to email my PMs – “why don’t we have a more prominent upsell for flowers, especially on the home page for V-day?” That mindset shift inspired me to actually test the main mission of my department in terms of my “buying experience.”

uTest: It seems that knowledge transfer must be especially difficult in the testing space. As someone going through the “on-ramping” process at a big company, with lots of moving parts, what advice you have for other testers and managers in the same situation?

Read more…

While social media is uber-important to us here at uTest, it plays a huge role when building any successful online community or crowdsourcing business model. The social web has led to a dramatic increase in online platforms that solve real business problems and enable social and business connections.

With all that in mind, uTest — along with HP, Dell, Microsoft, AT&T, FedEx, JetBlue, Samsung and others — was among the very first businesses to showcase their products on LinkedIn’s new Company Product Pages (launched yesterday 11/2/10).

LinkedIn has more than 80 million members worldwide and launched Company Pages to make it easier for businesses to engage with all these professionals with a dynamic, content-rich profile of record. In addition to product and service recommendations, businesses can display videos, featured product information and targeted ads on their Company Pages to drive engagement and interaction.

It’s still in the works, but check out our company services page here and recommend us if you feel inclined!
(And be sure to Follow Us On LinkedIn – it’s the only way into the ‘in crowd’ .)

If it can go wrong, it will go wrong (and at the worst possible time). This famous dictum from Murphy’s law should resonate with the greater software community – especially testers – as illustrated by the following videos.

First, let’s say you’re the CEO of a company that specializes in video-conferencing software and you have been invited to discuss your products on a video-conference interview for a popular cable news program. What’s the worst thing that could happen? Think real hard and then fast forward to 1:20 in the video to see if you were right.

Of course, this was certainly not the first (or worst) gaffe of its kind. Unfortunately, it happens to the best of us:

Read more…

Pardon my American “gun-loving” nature while I start this blog post with an analogy about guns. Let’s say that I’m interested in learning how to shoot a gun, so I go visit my local shooting range for instruction. The people who run the shooting range (which is fully licensed and very reputable) give me extensive safety training before finally issuing me a gun with some ammunition. While standing in front of the target, I carefully load the gun, point the barrel at my foot, and pull the trigger.

So whose fault is it that I just shot myself in the foot? Is it the shooting range’s for not issuing me special bullet proof boots or for not training me even more thoroughly about the dangers of shooting my extremities? Or is it mine, for shooting myself in the foot?

A recent Windows “bug” has put Microsoft in the position of the shooting range. I put “bug” in quotes because it’s hardly the shooting range’s fault that I shot myself in the foot, but it’s still their problem in the end. Same with Microsoft who must now deal with an annoying flaw in their DLL loading behavior that isn’t their fault but has become their problem. Ars Technica explains the problem very well, but here’s a quick summary of the issue:

Read more…

According to tech analyst firm IDC, U.S. companies paid a record $14.2 billion for paid keyword-driven contextual ads in 2009, with Google dominating 55% of that revenue, Yahoo 9% and Microsoft 6%.

More dollars = More fraudsters. Period.

The company Click Forensics just released a report on the overall click fraud rates for the paid search industry. According to SearchEngineLand, the report said click fraud was up from 17.4% last quarter to 18.6% in Q2 of 2010. Traffic across 300+ ad networks is reflected in the data.

In addition, it was found that the countries outside North America with the greatest volume of click fraud were Singapore, Pakistan, Japan, Ukraine and China respectively.

Recent research by marketing intelligence company Visual IQ came out with similar numbers earlier this month. The company estimates marketers lose an average of 16.7 percent of their pay-per-click budgets to fraud.

So why is click fraud slowly trending higher and higher? The CEO of Click Forensics, Paul Pellman, stipulates that “the main reasons appear to be the continued sophistication of botnets and malware prevalent in the fast-growing search marketing space.”

According to Inc. Magazine, click scams use the following techniques:

• Manual clicking. Workers might be paid to click to run up totals.
• Software clicks. Automated clicks.
• Bot networks. Using malware to harness unsuspecting users’ computers, criminals can create large networks of computers employing programs that imitate clicks.

Despite detection innovations, click fraud rates show no signs of slowing. Attacks are becoming more sophisticated. Criminals are making more money. So what can we do? Any advice out there on how to mitigate it?



The mobile wars are heating up! Microsoft is aggressively luring app developers for its Windows Phone 7 OS, while Android quietly gains market share. Blackberry expects big things out of OS 6, while The Big Apple deals with antenna issues, the yellow screen of death and the (remote) possibility of a recall. Interesting times indeed.

As part of our newly-launched “What Do uThink?” series (more on this shortly), we decided to ask our community which mobile OS they considered to be the best. Here are the results:

1. Android – 38%
2. RIM Blackberry – 28%
3. Apple – 16%
4. Symbian – 12%
5. Windows Mobile – 6%

“What do uThink?” is a weekly poll, where we’ll be asking the uTest community their preferences and feedback on various apps, operating systems and other technologies. To encourage voting, we’ll be awarding monthly and quarterly prizes to randomly selected participants. This quarter, for instance, we’re giving away an iPod Touch. The weekly polls open every Tuesday afternoon and voting takes place in the uTest Forums available to registered testers) as well as on our Facebook page. Got it?

Good. Now back to the mobile OS results…

Read more…

Take that Oracle! You just let Apple capture the lead in the 2010 Bug Marathon, otherwise known as Secunia’s Half Year Report (PDF). Worth the read, the 20-page report identifies the ten largest vendors with the most vulnerabilities (in all their products) and ranks them for the first half of 2010 – great entertainment for those who like to track bugs and keep score.

I mean, the World Cup is over and nobody really cares about baseball until September, so perhaps this could help fill the competitive void in the meantime…

Here are the current “standings”:

1. Apple
2. Oracle
3. Microsoft
4. HP
5. Adobe Systems
6. IBM
7. VMware
8. Cisco
9. Google
10. Mozilla Organization

As noted earlier, this is really more of a marathon than a sprint, so it would be useful if we went back a little longer than six months to crown a winner. Thankfully, Secunia did just that as part of their key findings:

Read more…

In the second part of our Testing the Limits interview with James Whittaker, we tackle Google vs. Microsoft; dogs vs. cats; why SCRUM is just a name; his advice for college graduates; bad habits of exploratory testing and more. If you missed Part I, you can find it here.

If you want to read more of James’ work, bookmarking the Google Testing Blog would be a good place to start. You can also read his 2009 book Exploratory Software Testing or check out some of his uTest eBooks and webinars.

uTest: The Microsoft vs. Google battle continues to play out very publicly in the media. Just last week, Computerworld wrote this story: “Microsoft: No Matter What Google Says, Windows Is Secure.” Having been at both companies, we think you have a unique perspective on this one. Any thoughts?

JW: Let me say right away that I enjoyed my time at Microsoft and admire the company and the smart people who work there. As a resident of Seattle, it is in my best interest for Microsoft to prosper! But the two companies are vastly different regarding the way their talent is managed and their products are built. Google is an engineering-centric company where innovation comes from individuals who are empowered to do whatever they need to get ideas into production. Much has been made of Google’s game-theory approach to managing people where rewards are given quickly for impactful behavior. It works. Morale is high and people work very hard and take quality very seriously.

Does this mean we produce more secure or more reliable products? We try hard to do so; Microsoft tries equally hard. I think we have the advantage of less legacy and a more modern and reliable platform (the Web as opposed to client operating systems) to work from. But the secret sauce at both companies is the same: hard work and due diligence.

uTest: You shared with us (as the pioneer of Testing the Limits posts) that your first assignment at Google was “To raise the level of testing precision and diligence.” So, how did it go?

JW: It didn’t take long. Google was mostly already there so I can’t really take credit for it. Now I am busy raising the bar further.

uTest: Top tester Glory Leung is curious: What are your views on SCRUM testing in general? Are people doing it properly? What is the ideal situation?

JW: Scrum is just a name. I don’t like names, they feel too confining and people have their own ideas of what they mean. I took a lot of flak for using the name ‘exploratory testing’ for my book. People love to confine you to how they view a specific named idea or technique. Flexibility is required.

Read more…

It was one year ago (June ’09) that James Whittaker helped us christen our ‘Testing The Limits’ interview series by being our first guest. And for much of the year, he held the distinction of generating the most page views… and then some guy named Patrick Copeland came along and took the lead a few months back.

Well, in honor of our one-year anniversary, James has accepted our invite to be our first-ever return guest – and this marks the start of a new trend. In our 2nd year of Testing the Limits, we’re going to be revisiting some of the past legends and leaders to see what’s changed since they last spoke with us. Of course, we’ll also be blending in some voices we haven’t heard from yet (we’re looking at you, Cem Kaner and Elizabeth Hendrickson) so stay tuned!

In this interview, James discusses his present role at Google; the emergence of Web Test Framework (aka WTF); the next decade of testing innovations; cloud computing and much more. When you’re done with this one, go read Part II.

uTest: A year ago, the big news was about your move from Microsoft to Google. Now that you’re no longer a Noogler, how has this year changed your perspective on testing and the testing industry? What has surprised you most?  Can you share any favorite stories?

JW: Four years ago I made the decision to leave the comfy confines of academe and consultancy and do something more real. It seems there is a steady supply of ex-industry folks going into consulting and not much of a flow the other way. I thought it would challenge me more than anything else I could do. Unfortunately, Microsoft just wasn’t the place to pull that off, ship schedules in the client-server domain simply didn’t allow a fast enough pace to suit me. I’ve been part of more software development in the past year at Google than I had my entire time at Microsoft and my consulting career combined. Things I didn’t think possible like shipping a product from concept to production in a matter of weeks, doing software development in a way that makes testing mostly invisible and creating completely new uses for test techniques that I had dismissed as amateur earlier in my career (e.g., record and playback) have not only surprised me but also now make my days a lot more interesting.

Another perspective that has changed is my appreciation of automated testing has grown. I’ve written extensively about manual testing and the importance of having a brain in-the-loop and I haven’t given it enough credit to automation in the past. Automation is really important and I think the detractors to it, simply don’t know how to do it right or simply don’t have enough experience with it. At the same time my appreciation for manual testing has grown as well, but I no longer advocate doing it without a lot of automated assistance. I’ll explain more about that later.

uTest: In the spirit of “WTF”, can you tell us more about the new, appropriately named, Web Test Framework and the unique control that Chrome and Chrome OS will offer web apps, browsers and the operating systems they are running on?

JW: I work with a developer who believes that WTF (the real meaning of the acronym) is the only appropriate response to a tester who creates yet another test framework. I have to admit, it is a response I often employ as well. Does the world really need another test framework? What the —-?

Well the world needs this one.

Read more…