In Part I of this month’s Testing the Limits series, Jonathan Kohl talked the life of the Agile methodology, his relationship with and advice about mobile app testing, and how new and season testers alike can advance their skills and feed their passion.

Today, he’ll talk about the biggest hang-ups teams have when it comes to testing, how to overcome those hurdles, how he became a fan of “gamification” in testing and a few other words of wisdom.

********

uTest: When speaking or consulting on mobile application testing, what’s the most common question you encounter and what’s your answer?

JK: This is the question that comes up the most after people have heard me talk about testing on mobile projects:

“Is it true that we need to test on real devices, incorporate movement, and test out in the real world, outside of the office?”

My answer: “Yes.”

Mobile technology has been developed to support movement, and it has a lot of dependencies. For example: networking (while moving you change networks, technology, and encounter lots of errors or weak signal conditions), location services (all about movement and spending time in different locations), weather and environmental conditions (temperature and light have an enormous effect on how some apps work), and movement (the devices have movement sensors, and are interacted with using touch screens and voice control. Combining inputs and sensors getting triggered from movement can be tough for apps to handle.). I know of no tool that helps us sort that out without getting physical with the devices. Hopefully we get better ones soon that focus on mobile technology, instead of treating it like a small PC/web browser. 

uTest: You work a lot with startups, when there’s a shoe string budget how do you communicate the importance of testing and QA?

JK: A successful business requires execution on several dimensions, for example, understanding your customers and their needs, having a great product or service offering that solves real problems for people, having enough investment to enable you to deliver, and a strong grasp of your financials. These require different skills and focus, but there is an over-arching need to create value for your customers, and part of that is great customer service. Technology fits directly into that, especially as we view organizations through the window (screen) of a smartphone or tablet. A poor technology experience is another type of poor customer service, so there is a direct line there. If you let down your customer or provide them with a poor first experience (which we are finding is increasingly on mobile devices), you can lose them forever. So not only do we need to have great people skills and a good strategy for satisfying and impressing our customers in our human interactions, but in our technology interactions as well.

A lot of people don’t realize that we are evaluated on our ability to deliver on the technology front, and that technology is a major part of customer service. It fits into the customer experience, but also in our offering of a product or service. Does the technology help, or hinder? Do we have an awkward experience, or is it seamless. It also has an effect on investment and financials, even if it is indirect. A quality product and technology stack is vital to the success of a business over the long term, so we need to invest in ways to make sure that we meet our customer’s needs there as well. Good quality processes, development and testing are part of that, and we ignore them at our peril.

uTest: Part of your consulting work is to help teams adjust to methodology changes. What’s the biggest hurdle these teams need to overcome and how do you help them do that?

JK: Dealing with people issues – different individual people with different agendas, motivations and fears. On one hand, you have people who have bought the marketing of a methodology and think that it will cure all ills. On the other, you have people who are deeply threatened by the change the methodology represents. It is a difficult balance to reign in the exuberance and unrealistic expectations of one group, while encouraging the laggards to get on board and actually try it out. I spend a lot of time myth busting on both groups. The methodology will not make everything perfect (and it won’t work forever), but it will also not cause you to lose your job, or cause bad things to happen to you. I work with teams by doing – getting in there and demonstrating, and helping address their fears. If I can do it, anyone can do it.

It’s also important to help them be able to measure the success of their methodology adoption in real terms: better product, better customer service, saved costs, etc. Just measuring success on adopting a new process misses the point, and will lead to some difficult situations down the road, particularly if that new process isn’t helping you create value for you, for your team, for the company, and for your customers.

People issues are hard to cope with. They aren’t fun, and it means we have to confront people on difficult topics. However, healthy disagreement and healthy confrontation are good, healthy things. If we suppress conflict, it just goes underground. We reach peaceful solutions through negotiation and co-operation, not by suppressing them in a need to “be positive.” There is also nothing quite like trying something out and seeing whether it works or not. Evidence, generated by a team that is committed to getting results, not just in implementing a methodology change, is powerful, and helps build teamwork and consensus towards reaching goals. Talking is one thing, doing and getting feedback on how we are doing helps enormously.

Read more…

Our Testing the Limits guest this month is Jonathan Kohl, a consultant and technical leader who writes and speaks on a wide range of software development and testing topics.

In this interview we talk to Jonathan about his passion for the field, what’s changed over time, his take on mobile app testing and his advice for new and seasoned testers. To learn more about Jonathan visit his website or follow him on Twitter @jonathan_kohl.

********

uTest: I think it’s safe to say that you’ve jumped into the world of software development and testing whole-heartedly. What drew you to this field and how do you stay passionate about it?

JK: The bottom line: I love to build things, and creating great software with a talented team is incredibly rewarding. Knowing that we have created something that helps make people’s lives easier is gratifying. I have found an ideal mix of problem solving, technology, creativity and satisfying and impressing customers in the software development industry.

I have great friends in the industry, and we keep each other sharp, and work on side projects together. We can talk about the latest technology, tease each other about programming tools we like and dislike, or branch off into just about any topic when we’re together. We also serve as a great support network for each other when someone is struggling, either technically, or personally. When we are working together to solve a difficult problem, nothing describes that energy of collaborating and working past your weaknesses, and that triumph of shipping working software at the end.

I ended up in this field by accident – I got lost on my way to law school. In the mid 90’s, I had three university professors who were incredibly influential: Dr. Michael Kubara (Philosophy), Dr. John Rutland (Management) and Dr. David Cowan (Mathematics). I was taking logic and philosophy classes from Dr. Kubara, and he told me to talk to a Math professor because I was delving into territory that he didn’t have expertise in. Dr. Cowan was teaching me Linear Algebra at the time, and jokingly said: “What are you doing with this ancient logic when the real research is in computer science?” Dr. Rutland had us study Data General for his class, and I was reading Tracy Kidder’s book The Soul of a New Machine.  The quirky nature of the technical people in the book appealed to me (I was playing in a band at the time, and read legal journals and philosophy papers in my spare time, yes, I was pretty quirky myself) and I loved the elegance of logic, and the questioning nature of philosophy. I agreed to try out a computer science class, and ended up getting sucked into an internship at a software company not too long after learning to code.

The software company reminded me of what I had read in Kidder’s book Soul of a New Machine, particularly the interesting, intelligent technical people. I enjoyed hanging out with them, and working on really difficult problems. I loved the fast paced, work-hard, play-hard culture of a software startup. I’ve stayed here ever since, with a brief trip back to university to finish off my undergrad degree. Once in a while I am tempted to pursue the law degree again, but I am invariably sucked back into a great project or opportunity.

uTest: You were an early adopter of Agile and later started talking about “post-Agilism.” What are your thoughts on the Agile movement over the years? (Since you could probably write an entire book on this topic give us the dust jacket version.)

JK: I started looking into Agile methods when I was working on a team in 1999 that was trying a combination of Rapid Application Development (RAD), Open Source, and iterative/incremental models like Spiral. The lead developer sent us this article called “Chrysler Goes to Extremes” which described the very basics of Extreme Programming. We liked that it was open and free (we didn’t have to purchase some tool chain and coaching we couldn’t afford) and we started trying some of the ideas out. We then bought the book “Extreme Programming Explained” by Kent Beck, and started implementing XP as best we could. A year later I was introduced to Scrum, and we had good success with it. After a while, I started noticing failures on Agile projects, and people moving on and doing other things. That was fascinating to me. Sadly, instead of learning why there were failures, there was an overwhelming urge to suppress them, or worse, dehumanize people by blaming them for doing it wrong.

As for Agile methods in general, I am ambivalent.  I am glad that lightweight methodologies are much more common place than they were in the ‘90s, and we have benefited from creating a common language around practices, and our tools are so much better now than they were ten years ago. I enjoy working on many Agile projects, since they fit my process and personality, and how I like to work.

However, there is a lot of snake oil out there, with proponents claiming that merely adopting Agile methods will lead to a successful business (What about having a great product, great customer service, skilled people and strong financials? Don’t those sort of matter too?) That really turns me off.  Furthermore, for years, any Agile failure would inevitably involve blaming the victim: you did it wrong, you don’t get Agile, if you were really Agile it would have been a success, which often sounds like another variation on the “no true Scotsman” argument fallacy. Sure, sometimes people fail because they made some mistakes, or didn’t commit, but every Agile project that fails can’t be blamed on the people.

Read more…

Our Testing the Limits guest this month is the one and only Fiona Charles. Fiona currently runs Quality Intelligence Inc., a small  independent business offering consulting services in software testing and test management. Prior to, she worked for companies like IBM and LGS Group as a consultant and test manager. Fiona is known as one of the most insightful speakers and writers in the testing space today – and we’re delighted to have her as a guest.

In this interview, we ask Fiona about testing soft skills vs. hard skills; how to rescue failed testing projects; the pointlessness of standardized documentation and many other topics. We hope you enjoy!

********

uTest: Your workshops and presentations cover an amazing range of topics – everything from “soft skills” like leadership development and the role of testing, to “hard skills” like risk-based testing, scenario-based testing and others. In your experience, what areas are testers more interested in learning about? Hard skills or the soft skills? And which do you prefer to teach?

FC: What do testers prefer to learn about? That depends on many things, including how they already view testing and on where they are in their careers. New testers are hungry to learn techniques (programmers, too). They need to go all out till they become journey-persons in their craft. Then, for many, the urge to gain a broader perspective emerges later when they’ve conquered technique. They become interested in project issues that they didn’t notice before and they begin to see that mastery of their craft requires additional, different, sets of skills.

I don’t draw much of line between so-called hard and soft skills. The testing topics that are compelling to me, which are therefore the topics I teach, are all loosely grouped around ways of thinking and ways of working with people. If you look at risk-based testing, for instance (at least for business systems), testers need to know not only how to think about technical risk in relation to components of a software system, but also how to identify the system’s stakeholders and ask them questions that will guide them to think about the real business risks posed by potential issues with that system.  Testers need to know how to organize and structure the results so that they drive a test strategy and feed into test design. So although I choose a specific topic area to focus on in a given workshop, I don’t feel that you can really separate the people part from the thinking part.

I’m eclectic in my teaching interests. I get excited about something and then I start thinking how could I teach that? How could I get people working together to discover new things or to look in a new way at things they already know, so they can learn from each other? And then of course, because I’m a consultant and I like to eat regularly, I start thinking about a conference where I could teach that, and whether it would be a good offering for a client.

uTest: As a consultant who’s worked with companies of all shapes and sizes, you’ve probably met every type of tester out there – the good and the bad. Based on that experience, briefly describe the perfect tester: What are some of their qualities, skills and habits?

FC: My list is quite long. These are in the top part of it. (I’m taking technique as a given.)

• courage
• empathy
• curiosity
• endless capacity for, and interest in, learning
• clear thinking and benign skepticism
• critical thinking
• creativity
• genuine love of software

uTest: We were interested to learn that one of your specialties was rescuing testing on troubled projects. What are some of the traits of a “troubled” testing project? And what are some of the more common remedies?

FC: There are troubled projects—i.e., projects where the whole initiative is in trouble—and there are what I think of as failing or stalled test efforts. You might see the latter as a consequence of the former, or the testing itself might be in trouble in a place where the rest of software development is supposedly running ok.

In my experience, the classic troubled project is late (sometimes by years) and mired in poor quality that people keep trying to ignore or hide. The testers feel they can never make any headway because there are so many bugs they can’t see anything else, and their management is continually pressing them to work longer hours and test faster, and demanding to know when they’ll be done. I’ve usually gone into projects like this as the test manager on a project leadership rescue team. Sometimes the whole project management team is being replaced. There is a range of other situations, such as where management feels that “testing is taking too long”, or missing important bugs, or the team is not coming to grips with an important project.

In any consulting rescue, whether or not I’m going to stay and lead the team or coach someone else in the role, the principal remedies are similar.

Read more…

Few people have a more diverse QA/testing background than that of Jane Fraser, our latest Testing the Limits guest. Jane is currently the Test Director for start-up called Anki. Prior to, she managed test teams for the likes of Electronic Arts, Vodafone and several other companies. You can read more about her extensive background on LinkedIn.

In this interview, we ask her thoughts about becoming an influential tester; tips for test managers; essential skills to acquire and much more. Enjoy!

********

uTest: This Spring at STPcon, you’ll be presenting on the topic of becoming an influential tester, in which you focus on developing several non-technical skills. What are some of those non-technical skills that a tester needs to become the “go to” tester in their organization? 

JF: Three things in particular:

1. Communication Skills: You need to be able to converse with others, and be understood. You need to be clear. “Doesn’t work” isn’t clear.  “When you press A, B is suppose to happen, but C actually happens” is clear.

2. Ability to understand their audience & customer: We need to listen and learn what is important. To a VP of Finance trying to convince him that we need to change a color a product because customers won’t like it, generally doesn’t get you any where. But phrase it in terms of sales, and that green will sell better than blue because of “some reason”, you have a better chance. Take time to understand their motivation.

One of my early lessons was at Corel, I found importing of graphics slow. And I logged a bug that importing was too slow.  Needless to say it came back – will not fix. I tried sending it back, again saying it was “too slow”. Back it came again.

A day or so later, our VP was giving us a presentation on the product and where we were going. He compared us to our competition several times. So now I knew he cared how we stacked up to others. Back at my desk I fired up the competitors programs, pulled out a stop watch  and 6 different graphics in different formats. I created a table with the 3 programs and the results. One example Competitors: 2sec, .5 secs.  Ours: 19sec.  The bug was addressed. I now tell this story quite a bit. Comparisons can be key, and knowing what is important to your audience (those making the decisions).

3. Ability to tell a story: Sometimes saying this happens when I do this, isn’t enough. But to tell a story about how it affects the customer. When a customer tries to do A, 50% of the time they won’t be able to.

uTest: Of course, testers can become more influential on an individual basis, but what are some the things preventing the department from becoming more influential within a business? 
JF: When the department becomes a team, and puts the product ahead of themselves. When your aim is to help others, great things happen. View your developers and producers as customers. We are there to help them achieve the common goal. Building relationships with them. When people understand that you care about what they are doing (creating a software product) they are more likely to listen.

uTest: Surely you have some awful horror stories from when testers (and test teams) were not taken seriously. Care to share any specific ones? If not, what are some of the big dangers in having a non-influential test team? 

Read more…

Last week I launched a mini series called Why the Testing Experts are Right. I take an excerpt from a past Testing the Limits interview and drum up some real life examples to prove that these testing experts aren’t all talk.

Last week’s installment featured a quote from Michael Bolton on “corner cases” and discussed why “it’s a fringe use case” isn’t a good reason for missing a bug. This week we’ll look at a quote from cyber security expert Richard Stiennon and discuss the special skills of a great security tester.

“Security testing of software throughout its development cycle is indeed different than quality and functionality testing. Instead of testing against end user use cases you have to have a mind set of an attacker, a completely different use case. In addition to meticulous use of security testing tools (HP-Fortify, Veracode, etc) a security tester must understand the application and how an attacker would leverage built-in functionality to subvert a system. A security tester must be diligent and detail oriented as well as imaginative and wily – a rare combination.” – Richard Stiennon

The first thing that comes to mind is the recent hacking of the New York Times. Though it’s not certain how their network was hacked, it’s speculated that an NYT staffer fell for a phishing scam. Here’s Kim Zetter, a senior reporter at Wired who covers cybercrime, speaking on NPR:

The New York Times story didn’t indicate exactly how the hackers got in, in its case, although they said that it might have done – been done through a phishing attack. And that’s often how a lot of attacks occur. A phishing attack is basically sending an email to employees, to workers and tricking them into either clicking on a malicious link, going to a website that has malware on it that’ll download to their system, or clicking on an attachment that installs malware on that computer. And that’s basically the initial doorway that they get into. And then they – from there, they route their way through the network to establish a more firm hold and install more tools.

While you won’t be able to stop people from clicking on malicious links, it’s important to have mechanisms in place that will further protect a system and detect when there’s an issue.

Even places that are supposed to be particularly secure still struggle with cyber security. The Federal Reserve confirmed that they were hacked earlier this month when attackers “exploited a temporary vulnerability in a website vendor product.”

There are lists and lists of companies and systems that have been hacked. Some had information stolen, some had their network effectively shut down. White hat hackers are continuously finding vulnerabilities large companies don’t know about or accessing sensitive networks that need to be better protected. But the thing many of these security breaches have in common is that the hackers found a way in that was unprotected. Software security testing is a highly specialized field. It’s different than finding a functional bug or evaluating usability. Security testers must know the common vulnerabilities, stay up to date whenever new tactics are developed and have the mindset of a hacker to stay one step ahead. Enter the rise of “white hat hacker” as a profession.

Read more…

You may be familiar with our Testing the Limits series where we interview thought leaders in software development and testing. These experts answer questions, give advice and share nuggets of wisdom that will help readers succeed in their careers.

How do you know they’re not just full of hot air? Well, for one, they probably wouldn’t be well respected members of the QA community if they were just lying all the time. But also, real life backs them up.

In this series, we’ll take excerpts from Testing the Limits interviews and pair them up with real life situations that prove these experts know what they’re talking about. There’s our first expert excerpt:

“Real testing, to me, should be based on investigating how the software allows people to deal with what we call ‘exceptions’ or ‘corner cases.’ That’s what we call them, but if we bothered to look, we’d find out that they were a lot more common than we realize; routine, even.” – Michael Bolton

This is what in-the-wild testing is all about. Ultimately, real life people will be using your software in real life situations. Sometimes, those real life situations are scenarios you couldn’t test for in the lab, didn’t occur to you or that you didn’t even dream of. But odds are your users will find them. While a crazy fringe use case will pop up from time to time no matter what you do, it’s important to take a step back, think through your testing and make sure you’ve come up with as many use cases as possible.

One of the best, most obvious and repetitive examples of “corner cases not really being that uncommon” can be found in video game development. Gamers are dedicated people and will try to make your software do things you wouldn’t believe. The biggest problems occur when they’re trying to do something reasonably within scope – uncommon maybe, but not unusual or unheard of. For instance, when Mass Effect 3 came out early last year the game encountered a bug that wouldn’t let players import customized avatars from earlier editions. If players set the appearance of their Commander Shepard avatar in the original Mass Effect and didn’t tweak it at all in Mass Effect 2, the character would revert to factory appearance when imported into game 3. From CNet:

Read more…

This was another great year for our Testing the Limits interview series. So to round out 2012, here’s a look back at some of our favorite quotes of the year.

January – Richard Stiennon

“Security testing of software throughout its development cycle is indeed different than quality and functionality testing. Instead of testing against end user use cases you have to have a mind set of an attacker, a completely different use case. In addition to meticulous use of security testing tools (HP-Fortify, Veracode, etc) a security tester must understand the application and how an attacker would leverage built-in functionality to subvert a system. A security tester must be diligent and detail oriented as well as imaginative and wily – a rare combination.”

February – Anne-Marie Charrett

“There’s a misconception that creating a visible testing structure is the equivalent of testing. This is not true. What makes a house a home? Not the roof or the external walls, but the people inside the house. It’s the same for testing. Testing is about the testers, their skill and discipline and how they interact and behave with others. That’s what companies need to focus on when setting up a team.”

“I think it’s essential that we take responsibility for the testing we do. This means each tester decides on their testing approach, what they test and when they’re done. Owning these decisions is what matures a tester, helping them become skilled, confident and motivated to excel in their testing.”

March – Testing Roundtable

“Real testing, to me, should be based on investigating how the software allows people to deal with what we call ‘exceptions’ or ‘corner cases.’ That’s what we call them, but if we bothered to look, we’d find out that they were a lot more common than we realize; routine, even.” – Michael Bolton

“Pretty good testing is easy to do (that’s partly why some people like to say ‘testing is dead’– they think testing isn’t needed as a special focus because they note that anyone can find at least some bugs some of the time). Excellent testing is quite hard to do.” – James Bach

“Testing has to be an integral part of developing software and not a separate phase. When this approach is taken, product quality is owned by everyone on the team. It is easy to state, but hard to put into practice because of long standing preconceived notions that developers and testers are better kept apart.” – James Sivak

“An explorer who doesn’t know much about testing probably won’t explore very well.” – Cem Kaner

April – Gerald Weinberg

“To me, the biggest weakness is not considering software testing anything but a (barely) necessary evil. Testing is seen as something that could be done by a troop of monkeys, so serious testers are treated like third-class individuals. The lack of means of acquiring testing skills arises from this attitude, as do most of the other poor practices in the testing business. You treat people as if they are stupid, then they will wind up acting stupid.”

“You can be a great tester if you have programming skills. You can also be a great tester if you have no programming skills at all. And, you can be a lousy tester with or without programming skills. A great tester will learn what skills she needs to continue to be great, in her own style.”

“Years ago, I thought punch cards were the last word in input. I thought that when I mastered sorting on magnetic tapes, I’d reached the ultimate in storage devices. I once thought that 2000 decimal digits was a vast amount of central storage. I guess what I’ve learned by now is that there is no ultimate technology, and if there were, I’d never live to see it.”

Read more…

Paul Holland is active in the Context-Driven testing world and the newest Rapid Software Testing trainer for Satisfice, and here’s why, straight from James Bach himself:

“Paul has been a tester and test manager at Alcatel/Lucent for something like 17 years. In recent years he has been using and teaching Rapid Software Testing methodology within Alcatel. His practical experience is fabulous.”

What more introduction do you need?!

In this month’s installment of Testing the Limits, Paul talks about how he went from eager pupil to official tester of Rapid Software Testing,  why context-driven testing is the way to go and why traditional education needs to change.

If you want to learn more about Paul, follow him on Twitter @PaulHolland_TWN

********

uTest: You’ve been an well-known member of the testing community for a while now, but you just recently became a teacher of Rapid Testing through James Bach’s Satisfice. First off, how did you discover Rapid Testing and what do like most about the approach?

PH: To fully answer this question I will need to be long winded in the first part of the answer and give you some history of how I met James and how I became an RST instructor. Then I will be able to answer the second half of the question.

I initially met James when he was the facilitator at the first WOPR (Workshop on Performance and Reliability) held in New York City in October 2003. At that time I was just becoming aware of the context-driven community. I had been invited to attend WOPR1 by Ross Collard and I was VERY nervous to be in the company of James Bach, Ross Collard, Scott Barber, Rob Sabourin and many other experienced performance testers. I don’t think I spoke much with James, or anyone else, at that first meeting. I was simply amazed at how much I learned in three days.

The second time I met James was when he was facilitating WOPR2 six months later at Sun Microsystems in Mountain View. This time I had some conversations with James and I found out about his Rapid Software Testing course. About a year later, we had James come to Alcatel to teach his course. I became fascinated with the material and the exercises. Over the next few years James and I became friends and whenever I spent time with him I would ask him to give me some of his exercises. Then one day (in 2007, I think), sitting in my family room in the west end of Ottawa, Canada, James finished an exercise with me and then stated, “Well, that’s it.”

“That’s what?” I asked.

“I’ve given you every one of my exercises”, he said, “… and you’ve done well in all of them.”

Well, at this point you could have knocked me over with a feather. Here was James Bach telling me that I had done well in all of his exercises and that I had done all of his exercises (or course, he is frequently developing new ones – so I’m never really done J ).

I was feeling rather big headed so I asked James, “So when can I teach RST?” He surprised me with his answer which was essentially: “I think that you would be an excellent teacher of RST. Of course, you will need to practice all of the exercises, thoroughly learn all the course content and practice teach with me and Michael Bolton but I don’t see why you wouldn’t be able to teach the course.” We then briefly discussed the type of agreement that would be in place for me to teach the course.

I spent quite a lot of time over the next two years doing what James had laid out and I was granted permission to teach RST. I chose to only teach the course internally at Alcatel-Lucent because I did not want to encroach on James’ and Michael’s livelihood. In my last 18 months at ALU I taught the course 7 times in three different countries. It was a great learning opportunity for me and it really allowed me to step back and understand how I had implemented the RST methodology over the past 5 years.

So, what do I like most about the RST approach? I really like how the methodology adapts to any situation with no best practices and no firm rules.

The way that I implemented the RST methodology in my team at Alcatel-Lucent was so different to what James and Michael were teaching. A couple of years ago I actually felt nervous that Michael would be disappointed in me as I was describing the details of my documentation and team management methods to him.

Instead of being disappointed he was elated. He reminded me that the whole point of RST is to use what makes sense in your own situation and adapt it to work well for you. We do not teach a cookie-cutter approach that will work inadequately in most situations. We do teach testers to think for themselves and encourage them to do what’s right – not necessarily what the current process says they must do.

uTest: We understand that you’re also a proponent of the Context-Driven School of testing. As someone with years of testing experience – who’s presumably seen many methodologies come and go – how much staying power does this philosophy have in your opinion? Will we be talking about Context-Driven testing fifteen years from now? Why or why not?

PH: I think that the only way to perform good testing is to adapt the approach to your testing mission to your own specific environment or context. You will be missing some major testing areas without considering how your current situation is different to other situations in which you have been involved. There is no way to apply the same techniques and process to every testing situation and have your testing be efficient and effective.

I would love to be able to say that we will not be talking about Context-Driven testing in 15 years but I fear that we will still need to talk about it as there will still be a large number of corporations that are ignoring their particular different contexts and will still be trying to implement “best practices” throughout their company.

Read more…

It’s the last day down here at STPCon 2012, but the show is far from over. Rather than posting another live blog (as we did on Tuesday and Wednesday) we’re happy to share our latest Testing the Limits interview with Karen N. Johnson, who is on stage as I write this discussing the Discipline Aspect of Testing. Karen is an independent software test consultant and a frequent speaker at conferences. Karen is a contributing author to the book, Beautiful Testing released by O’Reilly publishers. She is also the co-founder of the WREST workshop and has published numerous articles and blogs about her experiences with software testing. To learn more about Karen, check out her site or follow her on Twitter.

In this interview, we get Karen’s thoughts on mobile app testing; the latest testing trends; data warehousing and other topics. Enjoy!

uTest: You’ve been in the testing/QA industry for awhile now, but you’ve recently become very well-known as a mobile testing expert. First off, how did you get interested in mobile testing and what do you like most about working in the space?

KNJ: Mobile testing captured my interest nearly three years ago. One reason was having a client with a need and a request for my help, and the second reason was that the mobile environment was the biggest technology change I’d experienced in awhile. It is not mobile per se that captured my interest but the change, the chance to work on something new and to solve new problems. Being in the mobile space early on coupled with the fact that I am a writer, public speaker, and teach software testing including mobile testing, I quickly became seen as a mobile testing expert.

But I test in other areas and technologies as well. I’ve been in testing a long time. I’ve seen technologies come and go over the years. Currently, I don’t only work on mobile projects and I wouldn’t want to be exclusively focused on mobile. It’s the variety that I’ve most enjoyed in my career, so expect to see me continually change and evolve. For example, I’m also interested in data warehouse projects and working with remote testers and distributed Agile teams.

uTest: As a testing consultant, what is the number one mobile testing challenge you are asked to help your clients with?

Read more…

For this month’s Testing the Limits installment we reached out to the Agile Dad himself, Lee Henson.

Lee is one of the featured keynote speakers at this year’s STPCon (in Miami October 15-18) where he’ll be telling the audience about real life companies who have successfully mastered effective testing in an “efficient manner without sacrificing quality.” When he’s not speaking, Lee is an Agile coach and Certified SCRUM trainer. Follow him on twitter or visit his website to get some helpful Agile insights.

In today’s Testing the Limits, Lee will take us through why he thinks Agile is better, what it takes to succeed, why certifications are a good thing, how to convince your company to switch to Agile and more.

********

uTest: We are excited that our CMO Matt Johnston will be speaking with you as a keynote at STPCon, where you’ll be covering Agile testing. Can you tell us a little bit about effective testing methods, why agile is often misunderstood and how it can be the most effective testing solution? 

Lee Henson: I am super excited to be presenting at the keynote session of STPCon with Matt. I am looking forward to both meeting with him and learning from one of the best in the business. My views on effective testing methods are often considered very interesting. I believe that the best test is the one that is developed before a line of code is ever written. Regardless of what field you are in, defining success should fall way before measuring defeat. Many people believe that in addition to doing more with less, that Agile means saving everything for the last responsible moment without any up-front lifting. This is simply not true. Agile requires the discipline to define reasonable acceptance criteria and the willingness of the team to assume a more cross functional role when it comes to testing what has been built. Through the use of testing automation, enhanced unit testing and a willingness to refactor poorly written code, Agile testing methods truly do shine above the rest. The results are more immediate and give the team and stakeholders more frequent opportunities to inspect and adapt.

uTest: At AgileDad, what are the top three points you make to organizations to push them to make the transition from traditional testing to Agile?

LH: a) Agile testing provides a more frequent opportunity to review the tests you have in place to best ensure that you have adequate coverage across the most critical portions of your application.

b) As requirements continue to evolve and emerge in the Agile process, Agile testing allows for ease of updating the test plan with tests that will grow as your product offering emerges and grows.

c) Agile testing makes both developers and testers better at what they do. It forces them to think outside the box and blurs the line for many when it comes to cross-pollination. Developers want to write better code to impress the testers and avoid having to frequently revisit and refactor code. Testers learn to better understand how code works and functions which increases their proficiency at writing meaningful tests. This increases the value of your product and organization.

uTest: With your expertise in GUI web development – what are the most common GUI mistakes developers make, and in your opinion, how much of a difference can usability testing really make?

LH: Developers often get stuck in a position where they forget the key role or persona that will be interacting with the system they are building. This causes developers to either build the product geared towards a role that is too broad or to an audience that is not relevant. Usability testing is just so critical when building GUI Applications. Great GUI testing reminds me of being a great chef. While the developers are micro focused on preparing the perfect recipe, all the end user wants is a prepared dish that exceeds their expectations when it comes to taste. We need to take more time to understand at a greater depth who our intended audience is and make certain that the recipe matches their tastes. I can point out numerous examples of where the interface was frequently tested and put in front of the customer yielding a high value result in record time. Great GUI testing will increase your bottom line!

Read more…