Software Testing Blog

In Part II of our interview with Google’s Patrick Copeland, we discuss the challenges of managing a global engineering team; rewarding developers with food pellets; the difference between a good tester and a great tester; and why some companies will never launch a high-quality app. By the way, did you miss Part I of our interview?

uTest: What are some of the challenges that come with managing teams in dozen (or more) countries, as you’re currently doing? How difficult is it to maintain control over the people, processes and products? And when do you sleep?

PC: “Maintaining control over people” <smiling and laughing like Dr. Evil>.

But that’s not how it works at Google. The truth is…our team structure is atypical in the industry. For one, we are a flat company with many Nooglers being a few steps below senior executives. The expectation is that people and teams are semi-autonomous. In this model it’s impractical for managers to be controllers. And regardless, I’d rather set up teams that are made of great people who can run their areas themselves. My focus is on helping teams to be effective. Managers at Google are generally judged on their ability to enable smart people to get things done. Many have 15 or more direct reports, introducing some chaos and reducing the time available to micromanage.

One way we get everyone moving in a similar direction is to use OKRs, it came to Google thanks to board member John Doerr back in 2000. John stressed the importance of setting overall company Objectives and Key Results that would help develop departmental objectives; in turn, individual OKRs for every employee would support achievement of team and company wide goals. In Q1 of 2000, we rolled out our first company-wide OKRs, which included “8 million searches/day” and “Select CEO.” We’ve come a long way since then.

uTest: A lot’s been made of the unique and friendly work environment Google offers its employees. Does this also apply to your engineers? Or are they handcuffed to their desks and given food pellets for every line of code written (like we do at uTest)? Seriously though, how does an open atmosphere lend itself to better software?

Read more…

In this month’s Testing the Limits interview, we’ll put Patrick Copeland on the hot seat. Patrick is the Senior Engineering Director for a promising young upstart named Google (we’re not familiar with them ourselves, but we’ve heard good things) where he oversees a global team of about 800 engineers. But this isn’t his first rodeo –  prior to Google, Patrick spent a decade at Microsoft, where he specialized in all things related to software engineering.

So what do you ask someone who’s probably forgotten more about software than we’ll ever know? Well, in this installment, we’re going to get his views on catering to a global base of users; his criteria for evaluating testers based on their “tester DNA”; the recent addition of our good friend James Whittaker; the challenges of launching new products like the Nexus One, as well as other tidbits from inside the GooglePlex. Stay tuned for Parts II and III in the days ahead.

uTest: What are some of the challenges that come with having a global base of customers and users? Are certain products noticeably more popular in some areas rather than others? And how does this affect your future planning?

PC: Yes, of course some products and features do better than others. Our approach is to do lots of experimentation and to release and iterate. We push bits to customers early and often, and then we listen and watch usage. Customers help us by “voting with their feet.” Popular features and products are improved, and poorly performing products are deprecated. With a big focus on innovation, we also need to “fail fast” and customer feedback helps us make those decisions.

Not surprising, our global customers have different demands of our products. We want products to “feel local” and we need to support features that may be unique to specific markets. For instance, in Indic based languages using a standard keyboard is difficult, so we develop strategies like virtual keyboards or category browsing for search. As we specialize our products for certain markets, it introduces more challenges for testing (eg. requiring special cultural knowledge). When we can’t find internal talent, community-based testing is an interesting solution to this challenge.

We base staffing and planning decisions on several criteria:

• Strategic: Maybe a new feature, but in a market with existing competition (like Android).
• Financial: Obviously Ads and Search, but we have several emerging businesses that are also getting important.
• Customer usage: For example, popular high-traffic applications like GMail.
• Legal or Compliance: Certain areas need to be prioritized high for legal reasons. For example, SOX compliance for CheckOut.
• Ability to Impact: We look at our capability and decide if investing testers in an area would have a significant impact.

uTest: A few years back, you were the keynote speaker at GTAC, where you said something to the effect that “the longer I’ve been in the business, the less I know about it.” How important is it for testers and developers (and those who manage them) to maintain this student-for-life mindset?

PC: Very. When I hire people I look for folks with a “testing DNA.” These are people who are great computer scientists at their core, but also are very curious, love software, and are passionate about test engineering. People who have those characteristics tend to pursue challenges and continue to learn.

Read more…

So far, our Guest Blogger series has demonstrated the incredible domain expertise of our community – and this post will be no exception. With us this month is uTester David Vydra. A resident of San Mateo, California, David is an Agile Tester for Guidewire Software (he’ll explain what they do in a bit). If his name sounds familiar, that’s because he is the co-author of testdriven.com – a very popular testing site covering developer testing, automated exploratory testing, model-based testing and more.

In this post, David offers us his “notes from the field”, where he’ll be addressing the role of  software testers; criteria for hiring testers; the experts he follows and more. We’re confident that you’ll like it. Enjoy!

I am thrilled to be invited to share my testing experiences with the uTest community. I hope my account will encourage more organizations to adopt the agile way and more testers to find fun and fulfilling jobs.

I joined Guidewire Software about seven months ago as an Agile Tester. We make core software for the Property and Casualty insurance carriers. If you have car or homeowners insurance, you may be serviced using our software. Right from the start, Guidewire has used agile development methods and credits a good part of its success to this philosophy.

Our applications are complex. It typically takes several months for a tester to become fully productive because there is so much domain knowledge and in-house tooling to learn. Our applications are enviably flexible, and each installation is customized to fit the specific needs of the insurance company. In order to empower the customization process, we provide a number of development tools including a custom java-compatible scripting language, an IDE, a GUI framework, and a screen painter.

Read more…

Sometimes new technologies can inflame old problems.  For example, consider location based social networks.  Many sites like Twitter and Foursquare make it easy to post both what you’re doing and your current location.  This is a great concept, and as technologies go there are huge possibilities for combining location information with social networking.  But there’s just one catch: if you’re out and Tweeting about it, then you’re probably not at home.  And that makes your home a perfect target for robbery.

To help people become more aware about the ramifications of announcing that their plasma TV is unguarded, a new site has appeared called Please Rob Me.  Using the magic of social search, they track various networks and then list the posts from people who are clearly not at home.  Of course, this has caused quite a stir online as many have wondered whether or not something like this is legal, ethical, or even right?

Read more…

Darkreading.com published an article yesterday about a new proposal that could hold software developers accountable for security bugs. Not the “my bad” type of accountable – the legal kind. With support from some high-profile public and private entities, the proposal would likely require developers to make their software free of the CWE/SANS Top 25 Most Dangerous Programming Errors before it’s shipped. Needless to say, such a measure would drastically affect the day-to-day responsibilities of testers.

Stanton blogged about the Top 25 list around this time last year, noting that although it was comprehensive, it lacked meaningful context for testers. It appears that his feedback was incorporated into the 2010 version. Writes Kelly Jackson Higgins:

SANS’ annual list had been criticized by security experts as more of a laundry list rather than offering a solution, but this year the list came with so-called “focus profiles” that broke the programming errors into groups based on categories of weaknesses and also provided mitigation information. The list is in order of priority this year, with failure to preserve Web page structure (think cross-site scripting) as No. 1, and race condition mistakes as No. 25.

Not surprisingly, the proposal has sparked a lively debate among industry participants – testers, developers and consumers. Here’s how the pros and cons boil down:

Read more…

Over in Barcelona at the Mobile World Congress, 24 of the world’s leading wireless carriers and mobile OEMs announced their plans to create the Wholesale Applications Community (WAC) — a unified platform which developers can use to build a mobile app once and have it run seamlessly on any handset, OS or carrier.  Among the impressive roster of backers are mobile heavyweights like AT&T, Verizon, Orange, LG and Sony.  Sounds like a utopia for mobile developers, right?  It could be… if it works.

There are more than a few skeptics, including Jason Kincaid (@jasonkincaid) over at TechCrunch.  As Kincaid states (with a bit of help from Google’s Andy Rubin):

If it sounds too good to be true, that’s because it probably is. Andy Rubin, Google VP of Engineering (and the man in charge of Android) has already shared his skepticism, saying, “There is always a dream that you could write [a program] once and [have it] run anywhere and history has proven that that dream has not been fully realised and I am sceptical that it ever will be“. To put it another way, this is a pipe dream from carriers looking to loosen Apple’s stranglehold over mobile applications and there’s very little chance that it’s going to work.

The reasons Kincaid thinks the WAC won’t work out include:

Read more…

QUEST, one of the top software testing conferences, will be held in Dallas this year (April 19-23).  And uTest is getting geared up and is thrilled to be a part of this conference.

In addition to inviting Doron to be a keynote presenter, QUEST features a week-long agenda packed with more than 100 opportunities for attendees to build new skills and prepare for the testing professions of the future.

From exploratory testing to test automation to security audits to crowdsourced testing,  QUEST will cover a wide range of testing topics that give attendees insight into the latest best practices and innovative approaches to testing today. To learn more, here’s a sneak peek at the QUEST Magazine.

Special Note: Members of the uTest community interested in registering for QUEST are eligible for

Read more…

Every day, software testers must put themselves in the shoes of their users.  Testers should always think about how their users – their customers, actually – are going to use their product.  Is the application workflow clear?  Does a particular bug interfere with the overall experience of the product?  Could the UI be more standards compliant or intuitive?

Good usability testing is both an art and a challenge.  ReadWriteWeb, a popular web startup blog, learned that fact the hard way this week after writing a post titled Facebook Wants to Be Your One True Login.  The post was about the partnership between Facebook and AOL to integrate AIM and Facebook chat. Pretty standard Internet news stuff.

But take a look at that headline – it uses both the words “Facebook” and “Login”. For Google that was enough to make this ReadWriteWeb article the number two link for those search terms, and after that all hell broke loose.

Read more…

VentureBeat has an interesting article about eBay’s announcement that they’re going to tap into their user base to test new features — a kind of opt-in, ongoing beta program for new features.  The title for this article:

eBay to Use Crowdsourcing to Test New Features, Starting with Streamlined Search

Those who know me well know that defending the purity of the term”crowdsourcing”  against misuse is a pet cause of mine (e.g. – Meet-ups are not crowdsourcing; online polls are not crowdsourcing; asking your Twitter followers a question is not crowdsourcing). But don’t worry… this won’t be another rant about the importance of definitions and how critical labels are.  Well, at least not about the word “crowdsourcing”.

Read more…

With roughly 24 hours left in the 1st quarter Bug Battle, here is a brief summary of the results thus far: Nearly 800 bugs from over 400 participants.

Remember, the software testing competition ends Friday, February 12th at 11:59am ET, and there’s nearly $4,000 in prize money at stake. So if you have not participated yet, seize the opportunity now by reporting at least one bug to qualify for prizes.

If you were not able to submit a bug, but have strong opinions regarding the usability and functionality of the major TV networks, you may also qualify by submitting a request for the feedback survey. Don’t forget, there are separate prizes allocated to testers submitting the best feedback.

Special Note: For those who have participated in this competition, be sure to share at least one bug in the uTest Forums. This is shaping up to be one of our best Bug Battles yet. Good luck the rest of the way!