Most of the time, if we’re referencing an interview here on the uTest blog, it’s generally one from our Testing the Limits series. Today will be an exception, thanks to an amazing interview with Bill Curtis on ComputerWeekly.com.

Bill’s central argument – and it’s not an argument you hear very often - is that “the pace of change in the IT industry is hampering software quality.” Not progressing quality, not enhancing quality, but hampering quality. Think about that for a second. When you’re done, take a look at a few of his key excerpts (in italics), followed by my commentary.

BC: “Every five years we have a major change in technology, so every time we get good at building quality code, we change the game. There are new technologies and new languages. You make more mistakes because you are on are on a learning curve. We need a discipline for adopting new technology.”

Couldn’t agree more! Consider the testing landscape from five years ago, 2008. Mobile apps were still very much in their infancy, BlackBerry was still a dominant player, HTML 5 had yet to make much of an impact, Flash was here to stay – I could go on. The point is that times change! Thus, the idea of a discipline dedicated to adopting new technology is indeed quite interesting and would certainly have value (and appeal) for QA teams.

The hard part, I suspect, would be in determining which technologies are worthy of such an exercise, as you would run the risk of teaching an entire department a technology or programming language that never fully takes hold. Also, who makes the call? The QA manager? The product team? The CTO? Obviously there would be a lot of details to iron out, but it’s an interesting idea nevertheless.

Read more …

Despite the United States Postal Service’s reputation as an old-school, clunky organization that doesn’t changed much the Service has been making some giant strides in software initiatives lately. Over the past few years USPS has been embracing new technologies with whole-hearted enthusiasm – and producing noticeable results.

For the longest time, USPS was shackled to waterfall development practices that resulted in slow, costly projects that often failed to meet user needs. The Service’s systems were disjointed and hard to use – resulting in a high chance of error and frustration.

Since piloting Agile projects in 2010, though, USPS has found a better way to complete projects in a timely, cost-effective manner. Even better, their efforts have resulted in improved customer satisfaction. USPS’ mobile app has a higher Applause Score than than Fedex and UPS. This dedication to user satisfaction has been a driving force behind the service’s new approach toward development. As part of the switch to Agile, the Postal Service began incorporating customers into its dev process. These potential customers provide feedback throughout the process and help the organization make sure the service its creating is staying true to customer needs. From FCW:

The [project] incorporated fast deliverables and daily scrum sessions between IT officials and customers to ensure parties were always on the same page. In waterfall approaches, “IT goes off and builds something that 12 months later might not meet the customer’s expectations,” Edgar said, noting that in times of austerity and rapidly changing technology, that approach rarely makes sense.

“The intent was to find a way to deal with changes where we could be more responsive to changing customer expectations and raising the visibility of what we were doing throughout the development’s lifecycle back to our business partners,” Edgar said. “We wanted to better partner with our business organizations, and maintain a more consistent and visible engagement between teams where we could prioritize requirement building so we get the most important things done first.”

Another reason USPS is finding success with its new Agile approach is the roll-out design. Instead of creating a new system as a whole and rolling it out to all customers, USPS has divided its client base for some projects into size-based segments. The project discussed int the FCW article was completed and rolled out to large businesses first to see how companies would respond. Now that the Postal Service is happy it’s beginning to bring the service to smaller organizations. This segmented, rolling approach helped USPS “complete” the project faster and more effectively.

USPS officials said they are very pleased with the results, especially in regards to cost avoidance. …

“What made agile so valuable to us was that it had a hard deadline, whereas if we used a waterfall methodology, we’d probably only just now [June 2013] be planning on doing something,” said Cathy Moon, who formerly managed MTE at USPS. She now works as a manager for USPS’ Operations Integration and Support.

Part of the reason USPS is finding so much success with this new development model, in my opinion, is the fact that they’re focusing on the customers. Sure, these shorter deadlines are helpful and give teams something to aim for and focus on, but a deadline is just a deadline. If the Postal Service hadn’t integrated customer interaction so deeply into the projects they very well could have ended up with fast turnaround projects that were just as disappointing and dis-jointed from user needs as ever. The focus on the customer is the real value-change here.

 Design teams work with customers “almost daily,” and communication between parties is integral to a project’s success. …

“Agile is a way we deliver solutions to our customers, and get those solutions faster and with a more successful conclusion to users, wherever they may be,” Edgar said.

Read the full article at FCW >>>

You’ve mastered the technical skills, but why aren’t you advancing in your QA career? Joel Montvelisky, a tester, test manager and QA consultant with over 15 years of experience in the field of Quality Assurance, tackles this question in the following guest post.  You can read more about Montvelisky’s views on testing, agile, training and more on his blog – QA Intelligence.

******

While presenting at a recent training session to a group of testers, one of them asked me what were the most important skills I thought a tester should have in order to advance in his career.

As I had my “mentoring hat” on, I immediately asked the whole group what they thought the most important skills for testers were.  They started throwing out all sorts of ideas, like analytical thinking, the ability to “read” code, knowledge of web and mobile technologies, automation, an eye for detail, etc.

I guess I should have expected that. Working as we do around programmers and engineers, the testers focused only on the hard and technical skills.  And I don’t blame them either, since these skills are incredibly important. In fact, I even wrote a post about being a technical tester in my QA Blog.

But in a sense, this is also one of the biggest mistakes you can make as a tester; to focus solely on the technical skills and not develop the “softer” skills that are also vital to performing your work well.

If you look at the testers who are the most valued by your company, in addition to technical skills, they also possess a number of other skills that help them to contribute more and to distinguish themselves and their work.  These are also the testers with greater chances of advancing to higher managerial positions.

Wouldn’t you want to be one of those people and have those opportunities yourself?

Well, I can’t sell you a potion for that, but I can explain the softer skills that are helping these most valuable testers, and how can you develop them too.

There are many soft skills a tester can acquire or develop, but after interacting with a large number of testing teams and development organizations, I’ve identified three skills that I believe are the most important for a tester. These 3 skills are:

1. Communication skills
2. Political skills
3. Customer-facing skills

Communication skills

Communication skills are the single most important set of skills a tester should have – even more important than any technical skill you can think of.

By communication, I don’t mean only the ability to write a clear test case or an informative bug; it goes much further than that.

Communication starts with the ability to listen to what other people are saying and to translate that information into action.

In a QA Blog post I published in the past called, “Of Testers & Soldiers” I likened the role of testing to that of an intelligence officer’s in the army whose task is to seek information from multiple and scattered sources, and then piece it all together in order to map out a complete plan of risks and actions for his commanders or managers.  This can only be done if you are able to both listen and process the information quickly and effectively.

An additional aspect of communication is the ability to transmit your message clearly and in a way that will encourage your audience to listen to what you have to say.  As an old colleague of mine once told me, “We are all salespeople.  Some of us sell cars, others sell ideas.”

Read more …

If you could go back in time, what advice would you give to yourself in the realm of software testing? Would you suggest developing a different skill-set? Would you rid yourself of bad testing habits? Would you advise avoiding testing completely?

Unfortunately, you can’t go back in time to give yourself advice; the universe would collapse on itself (duh). However, it *is* possible to give advice to the young testers out there, many of whom are loyal readers of the uTest blog. So today, that’s what we’re going to do. As part of a new series that we’re calling The Top Secrets of Top Testers, we’ll be asking a panel of testing experts, uTest community members and internal team members a new question every month. As you might have guessed, this month’s question is:

“If you could offer a new tester one piece of advice, what would it be?”

********

Jerry Weinberg – Author, Speaker, Consultant

My one piece of advice would be this:

Don’t give advice. Each person is different, so spend your time developing your own skills as a tester. Your advice probably doesn’t apply to others the way it applied for you (and that includes my own advice to you).

Jonathan Kohl – Writer, Instructor, Consultant

My advice: To learn how to work out software testing for yourselves.

Have you ever wondered what software testing actually is? Many people test, but find it difficult to explain. When I was writing my book: Tap Into Mobile Application Testing, I realized that some of my target audience would be new to software testing, so I wanted to make sure I made the topic approachable. It took a lot of analysis, reflection and thought to explain what it is that we do. In chapter 1 of my mobile testing book, I try to do just that, and I advise any new tester to:

• Use the software systematically,
• Observe what is going on carefully,
• Evaluate the effectiveness of it bravely, and
• Record and report anything interesting.

It can take a while to learn what sorts of problems to log, and how to spot issues. I start many testers off by reading the Design of Everyday Things by Donald Norman, because it is a great guide to spotting problems in things we use all the time. Many of us don’t notice there are issues, or we blame ourselves when there are problems instead of the technology. It takes time to develop the confidence to point out problems in software, whether they are design-related, due to computing errors, or obvious problems like crashes or performance issues. Testing consultant James Bach says that a bug is something that bugs someone who matters. You are a tester, and you matter, so anything that bothers you is worth noting. If it bothers you, it is going to bother a lot of other people. This requires courage, because people don’t always react well to our feedback, even when they ask for it.

Develop your skills in using software, and in finding ways to learn more about it, including looking through log files for important information, or using monitoring or other software to help gain more visibility into what is going on. Use the software from as many different perspectives as you can, and if something is confusing, or bothers you in any way, report it to the team and get that issue into the conversation. It’s important to be systematic, to be observant, and to follow your instincts, and learn how to spot problems and report them effectively to your team. That is one way to define testing, and the better you get at all of those areas, the more value you will provide for a software development team.

Read more …

The recent news stories surrounding the NSA programs that collect phone data and track overseas email have stirred up a lot of controversy. No matter what side of the issue you fall on, the take-away no one in the software application world should ignore is just how touchy people are about privacy and transparency. One of the main arguments being played out in the media about the NSA leak – and even mentioned by some top ranking officials – is that the public should have been given more information about the existence of these programs. Something can be confidential without being so secretive. This is the argument you, as an application developer, tester or brand owner, need to be paying attention to.

We’ve seen this issue come up time and again – users notice changes in privacy policies and start spreading the word about the “nefarious” practice. Petitions are created, people call for boycotting the offending application, rumors abound, false “security steps” are spread and users generally complain and call for a redaction of the change. This can be sparked by the smallest, most insignificant change. All it takes is a misunderstanding of the new policy or a “secret” change that users weren’t notified about. Catch users off guard or be vague and confusing and you’re asking for trouble.

Take January’s Instagram debacle for example. The service did notify users that a change was coming, but the updated policy was confusing and lead people to believe their photos could be used in advertisements without their knowledge or permission. Instagram scrambled to clear the air.

Earlier this week, we introduced a set of updates to our privacy policy and terms of service to help our users better understand our service. In the days since, it became clear that we failed to fulfill what I consider one of our most important responsibilities – to communicate our intentions clearly. I am sorry for that, and I am focused on making it right.

Even after that post, the policy left something to be desired in the “communicating clearly” category. When legal experts took a look at the policy, they were just as unsure about what it allowed the service to do – which doesn’t bode well for transparency. From The Washington Post:

But the updated language was not immediately available, leaving many users still skeptical about Instagram’s intentions. Legal experts said the “terms of use” document was remarkably expansive. Elements would apply to users as young as 13.

Privacy is still an evolving field and we’ll be creating new standards and best practices for years to come. But the one thing we can do now and continue doing as privacy standards change is work on transparency. If users are presented with changes and understandable terms of service and notified when an application is accessing or storing extra features or data they can make a more informed decision and are far less likely to feel blindsided and violated.

In a lot of situations, notifying users (in a clear, understandable way) about your privacy practices, what data is being tracked and how it’s being used is legally mandatory. The California Online Privacy Protection Act (which also applies to mobile apps) requires that “an operator of a commercial Web site or online service that collects personally identifiable information through the Internet about individual consumers residing in California who use or visit its commercial Web site or online service shall conspicuously post its privacy policy on its Web site, or in the case of an operator of an online service, make that policy available.” The policy has to notify users about “the categories of personally identifiable information that the operator collects … and the categories of third-party persons or entities with whom the operator may share that personally identifiable information.”

A lack of transparency (among other things) has landed Facebook in hot water more than once. In 2011 Facebook was forced by the FTC to adopt more transparent privacy practices – and to stick to them. From Macworld (emphasis mine):

Read more …

I always thought that those “Christmas in July” campaigns were bogus – that is, until I came into the world of software testing. While the holiday shopping season is probably the last thing on the minds of consumers, the same cannot be said of retailers and their respective dev and QA departments. For them, the holiday shopping season is well underway. And yes, I realize it’s not even July yet.

By this I mean to say that retailers (who, to be fair, never really have an off-season) are busy prepping and refining their applications so that they are ready to roll when the peak holiday shopping season comes around. For them, identifying and fixing bugs becomes a top priority during this time, as they do NOT want these types of issues appearing before users come November/December.

Why? It should be obvious to most, but a recent guest post on retail-digital.com frames the problem perfectly:

Approximately 70% of errors in software projects arise during the early analysis and design stages. Incomplete or inaccurate requirements and inadequate testing can lead to costly re-work, while Early Error Detection and correction can cut the resulting costs by up to 90%.

When software errors and system failures cause consumers to perceive a retail brand negatively, improving quality becomes a priority. However, hastily implemented quality programmes are not the answer. Quality-driven retailers have clear objectives and robust processes for gathering, analysing and taking action on information and feedback on a rolling basis.

Bottom line: Poor quality can cost retailers customers and money! To make sure this doesn’t happen, here are 4 tips for retailers when it comes to testing:

Read more …

You know you’re not going to meet your deadline, you haven’t allotted enough time for testing, the scope is poorly defined and constantly changing or there is a communication gap between your team and management.

Despite the cause, at some point all developers experience a roadblock which leads them to believe their project will almost certainly fail. Scott Ambler, senior consultant on software development and agile adoption, says that software development projects often fail because of an organization’s unrealistic goals for:

• “Scope (what must be built)
• Schedule (when it must be built by)
• Resources (how much it must cost)

The development team has failed at renegotiating the situation, and is forced to try to deliver under those constraints anyway.  In the end, if the project team delivers at all, the quality of the delivered product suffers and the project is almost always late and over budget anyway.”

But when you know a project is failing, can you salvage it before it’s too late? A recent post by ITWorld’s Phil Johnson, covers advice for developers whose project seems to be “circling the drain” so to speak:

“1. Share your concerns with the powers that be

Communicate your concerns in the most concise and non-confrontational way possible up the management ladder. Summarize the risks, but do not try to impose your conclusion on them.
MrFox

Only include facts and objective observations. Leave all conclusions up to whomever (if anyone) reads what you’ve written.
Dan Pichelman

Make a few polite suggestions for improvement. Don’t sound the warning bells, don’t be doom and gloom, just be polite and subtle.
cadmium

2. Continue to do your best on the project 

Try to provide answers, not just problems. Look like you are trying to fix them.
Ozz

Focus on personal excellence – strive to write ever better code, meet ever higher standards of quality and functionality.
code4life

Definitely don’t slack.
Philipp

3. Focus on what can you learn or gain from the situation

Oftentimes, the same chaos and disorganization that comes with a failing project, affords you the opportunity to shine. So look at the project this way: what opportunity does this failing project afford, for the light to shine on all of my strengths and best qualities? What lessons am I learning from this experience, that will make me a better professional and a better person?
code4life

This is your chance to practice important skills in systematic thinking and interpersonal communication. Understand and visualize the issues and potential opportunities that are being missed so you can develop a strategy to communicate these as clearly and simply as possible.
Tone

You can show some leadership skills here that might help you later along the way, treat the problem as a challenge.
Eran Medan

Read more …

Imagine you’re a software tester who just got a assigned to a project, and you’ve quickly discovered and reported tens, hundreds, thousands of bugs. This makes you an all-star, right?

Now look at it from the other side of things, from the perspective of the company or developer who is receiving those bugs. That mountain of bug reports probably feels like a lot of noise.

So, should testers report every bug they find? It’s a tough question, and one that has sparked an enticing debate within our community of 80,000 testers.

“No, It’s Too Much Noise”

Some testers say no, don’t report every bug you find. These testers claim it’s too much noise for the recipient, thus detracting from the high-priority issues. According to testing expert, Lucas Dargis, “I usually give [new testers] some unusual advice. I tell them to ignore low-value bugs; don’t even bother reporting them”. However, Dargis’ advice isn’t all that unusual.

It is quite common that the development team only wants the most relevant bugs reported. Dargis’ perspective comes from his own experience testing a buggy application and (regrettably) reporting all the bugs:

“I was working on a particularly buggy application. I took the approach of logging every bug I came across, even though I knew most of them were low-value. The bug list grew quickly. Soon I had over 100 bugs, but logging bugs takes time and so I had only been able to test about half the application.

It took our PM a week to weed through all those bugs and in the end, he decided that the majority of the bugs were not going to be fixed. Moreover, I had already used up my allotted testing time so we had to launch the product without full test coverage. As you might expect, our customers found several serious bugs in the app after it was in production.

So while technically I didn’t do the wrong thing (I reported bugs, that’s my job right?), I didn’t do the right thing. I should have focused on serious high-value bugs first and only after we were confident we had found enough of them, move on to the low-value bugs as time allowed. I could have saved myself and the PM countless hours of work, and we could have released a higher quality product.”

Low priority bugs can clutter and overwhelm the development team receiving and attempting to fix the issues. By only reporting bugs in-scope, that need the most immediate attention, dev teams can fix the big issues quickly. It makes a lot of sense.

“Yes, It’s My Duty As a Tester to Report All Defects”

But let’s look at it from the other side of things. A bug, no matter what the value, is a bug. According to testing expert, Ryan LaMontagne, as a tester it is your responsibility to identify and report software issues:

“…I believe it is our job and responsibility to file the defect(s). We as testers don’t know if this will be considered high value or low value, but it will deliver value the client expects.

I always tell my Team to never make an assumption. It’s better to file a defect than let it go. Let the client make the decision on the value unless specified.There is a cost involved with fixing defects later in the development cycle and I feel it’s our job to help the clients by reporting all defects, letting them decide on the value, and save them cost upfront.”

Read more …

In an ideal world, testing would be done within the comfy confines of a staging environment, where software bugs – no matter how big or small – had zero chance of making it in front of real users. In this environment, testers and developers would have ample amount of time (and practically no pressure) to identify and fix issues. They could then launch with no surprises and sing songs together while their bosses showered them with praise.

We don’t live in that world. Instead, we live in a world that sometimes requires us to test our applications in production, in front of real users. Sometimes, there’s simply no other option.

But what are some of those instances? When does it make sense to test in production vs. a staging environment? I don’t presume to know all of the times when this might be the case, but a recent article from the insurance industry (ironically enough) highlighted a few times when and why testing in production is necessary.

Here is their broad overview, followed by a few examples:

In fact, we test in production all the time. Even well-disciplined, well-managed application development teams must rely on a production release to fully vet and test their code. I am not talking about unit testing, or functionality testing, or integration testing, or system testing or even performance testing.

We still need to do all those things and if we do them correctly we won’t have any bugs or code defects when we go to production. But it is impossible to validate actual performance of a complex system in a lower life cycle. The key word here is test. You can test and retest, but all you are really doing is validating the performance and functionality of the application in a testing environment.

There are a few specific examples mentioned by the author – and they are some of the same reasons cited by TiP expert Seth Eliot when he was interviewed on our blog. The first of which involves data. The author writes:

Read more …

Last month, the uTest Community Management team kicked off the uPanel Webinars!

Rather just listening to one person talk in a straight presentation, the uPanel Webinars are very open and involved discussions on helpful topics for the uTest tester community. During each webinar, the panelists respond to questions from both the moderator and the audience.

In one of our most recent (and very popular) webinars, we explored the topic of video documentation for mobile bugs, something that developers are increasingly requiring from their testers. Video documentation helps to clarify the actions leading up to the discovered issue, and especially valuable with in-the-wild testing. Many testers struggle with determining the best method for capturing a bug in video on particular devices in a way that will give the TTL and the customer the most detail possible.

The video capture uPanel Webinar explored these challenges, and provided best practices for capturing video documentation of bugs in mobile test cycles. Kayla and Tara made some great recommendations for tools and tips to follow when trying to grab the right visual info for developers and customers. Check out the webinar below: Read more …