Thoughts on the Future of Software Testing
[ed: We are grateful that James has posted some additional thoughts for our community following his outstanding webinar. Make sure you watch the video of his uTest webinar on The Future of Software Testing.]
Kudos to the uTest community for making my webinar so successful. From my perspective it was the questions at the end of the talk that made it so valuable. It’s good to see my opinions challenged and it’s also nice to get the pulse of such a large and engaged community.
Clearly the subject of keeping testers in test and seeing they get promoted weighs heavily on the minds of many. On my MSDN blog, this has been a hot topic, but since the idea started here at uTest, I wanted to post my larger and perhaps more controversial opinion here.
I will not mince words and I will not pander to the test hard liners. Testers are and have been treated as second class citizens in the software development industry and sitting around whining about the glass ceiling is not going to get us anywhere. What we need are two things, innovation and a mentoring system.
Innovation has been seriously lacking in software testing. The test tools that the major vendors are shipping now aren’t much different than the tools they shipped a decade ago, yet look at dev tools and there is a fundamental change in the theory and practice of how software is written. There are new languages, new IDEs, new static analyses … the pace of innovation has been brisk. Who wouldn’t want to be part of that? When are we as a testing community going to step up and match that pace of innovation? Until we do, we’ll continue to lose our best and brightest and who can blame them? Let’s up the ante with real innovation and change the face of testing so completely over the next few years, just like developers have done over the past decade. Then we may even see those same bright testing minds coming back. When the pace of our innovation pushes the quality of our software to noticeably higher levels, that’s when we’ll get noticed ourselves.
Mentoring is the next aspect of this problem. We need to take this far more seriously. There’s a quote on my MSDN blog you should read about the frustrations of one of our best testers who sees dev talent promoted faster than test talent. Why? Because there are more senior devs looking out for the junior devs. They have sponsors at higher levels in the company whose coattails they can ride and who take an active interest in their careers. At Microsoft there are fewer such senior (we call them partners) folk and that means they have to work even harder. But I don’t see them doing that. Instead, they seem to hunker down, relishing their status and trying their best to keep it. I wish they were judged on their ability to identify and promote talent instead of being judged on their ability to maintain their own position. We need stronger people at the top of this discipline that can act as representatives for the rest of us.
This uTest community can help. We are large enough and can become strong enough to act in a concerted manner to instigate, incubate and propagate real innovation and real mentoring. Together we may be able to do what our forebears could not.
Ideas? Let’s hear them.
James,
I’m sorry I wasn’t able to stick around after your webinar to hear the comments. I’ll have to check and see if they are at the end of the recording.
As far as innovation, there are plently of new test automation languages and tools. As technologies such as Web Services become more important, new tools and techniques pop up to support their testing.
Perhaps mentoring isn’t big at Microsoft. I don’t know why that would be. But I see plenty of mentoring in smaller companies – at least the ones where I have worked.
So how does this all fit in with uTest? I know you feel this “crowd sourcing” is part of the future of software testing – I just don’t see it filling other than a small niche. As best I can see, this is just a paid, public Beta.
Public Betas are certainly not innovative. You post your Beta, provide minimal instruction, and hope for some sort of useful feedback from a variety of users. If you use uTest, you also pay a small amount. If you do it yourself, you give out some t-shirts, or i-pods, or free licenses.
And I can’t imagine you think there’s any mentoring going on here? Mentoring certainly doesn’t fit in with this “crowd sourcing” paradigm. I believe you used the phrase “You throw it over the wall to the crowd” or such in your webinar.
So we need more innovation and more mentoring, in order that testers not be treated as second class citizens. Ok. But didn’t you also say “Most apps don’t require dedicated QA staff”? That sounds rather second class to me.
You raise a couple of issues that I’m sure are important to you. But I don’t see how this fits in with your future of testing, nor with uTest.
-joe
James I attended the Webinar conducted by you.It was a very informative webinar and it helped me in thinking about how each tester should be equipped for the future by keeping pace with the technology.It has created a vision in me about how different testing can be ten years from now.UTest cannot actually offer mentoring.There can be discussion by prominent persons in UTest who can share their ideas through the discussion forums in UTest.This will help to Kindle thoughts in fellow testers all over the world
Anita
James, you are right about the need for mentoring. One of the best places I have found for test mentoring has been the Black Box Software Testing course given by the Association for Software Testing or BBST (http://training.associationforsoftwaretesting.org/)
I encourage anyone in the uTest community to sign up for a class. I am about to complete the Foundations class and it is phenomenal. Cem Kaner and James Bach have done a fantastic job of creating a course that is challenging, imformative, and relevant.
Brett
I second Brett’s thought that the training provided by AST is quite unique and unlike any other commercial software testing courses. I have completed the foundations as well as the bug advocacy courses and today feel much more comfortable and confident in my test efforts.
In my six years of experience in this field I have never got the kind of support, guidance and feedback that was provided by Dr. Cem Kaner, Scott Barber and other course instructors during these two AST Training programs attended by me.
James, I feel Dr. Cem Kaner has instilled some qualities from you during your association at Florida Tech and thus you both now seem to be the change agents for software testing profession.
Here’s the response I gave Joe for his comment above.
I think you missed some key points. Particularly the quote “most apps don’t require dedicated QA staff” the word is “won’t” because that is a prediction about a future that has not yet arrived.
I see crowdsourcing as a natural step in the evolution of sourcing. It’s different than a beta and better because it builds community in addition to providing testing services.
I am not sure how you got the idea that there was no mentoring going on at Microsoft. The problem is that MS has 90k employees and mentoring on that scale is a hard problem. Personally, I am doing a lot of mentoring. But we don’t have enough mentors and many qualified ones are too busy or aren’t good at mentoring. It’s a rare skill, I think.
I am also not sure why uTest is unsuitable for mentoring. Certainly it isn’t set up that way now, but there is no reason it couldn’t be and there is no reason to dismiss it so quickly. I think uTest is very innovative and I like seeing people like this actually DO something because the test community is full of people who complain too much about things not getting done. We need more doers in my opinion.
Finally a comment on the comment about education. I haven’t sat the AST courses but I have a lot of respect for Cem and James and imagine that if they are behind it, there must be some good material there. I feel that way about other consultants as well. There is some good training around, it just takes a lot of searching to find it.
There’s a lot of training you can do yourselves as well. Try this as an exercise:
Obtain the list of bugs that escaped your test team’s diligence. Maybe these are bugs found by uTest, or in a beta or in your released product. As a group, ask the following questions about each bug:
– What fault caused it (maybe you can invite a dev to help with this one)
– What were the failure symptoms that allowed you to isolate it (the answer to this will build bug spotting skills)
– What was the thought process you used to come up with the test case in the first place (this will allow you to codify your reasoning about making good bug finding test cases)
This exercise when done as a team event can be a huge learning experience that has direct application to your product and your productivity. Try it, you may find that the best education you can treat yourself to is something you don’t need to spend any money on at all.
I agree on the tools, however I experience that a lot of testers seem to shy away from tools that seem like “programming”. They need to get over this (Brian Merrick has worked on this). Also we do need more advanced and integrated test tools. There are places working on this.
On then mentoring, I am surprised about how many testers don’t seek mentors. I know of companies (mine) who have mentoring programs, but without insentives, minimal use is seen. On the industry side, we need to improve too, but it takes effort and again people often don’t extend the effort to find and stay connected, though there are many great forums (like this one).
Hi:
I am new to this community. I apologize if the questions I raise have already been asked and answered.
I also tend to agree with Joe that crowd sourcing might only fill a small niche. I think crowd sourcing falls under test execution phase of the Software Development Life Cycle Quality management. There is a lot more to testing such as:
- Capturing requirements in the form of test cases in a TDD methodology.
- Test data management, for data centric applications and record keeping systems.
- Risk based testing.
- and a lot more.
Once you treat testing as a quality management process and not just kicking the tires at the end, then it becomes complex and interesting enough to absorb top talent who would consider themselves as first class citizens. I come from a development and architecture background, and I am finding out that complexity of testing problems grow exponentially by extending existing apps with simple components. I think testers should have a good understanding of the underlying system and app architecture and internals of the system. So besides process mentoring, deep understanding of the system is necessary too. This cannot be done by someone who views him/herself as a second class citizen. Testing can only be successful by attracting top talent who can discover and mentor at the same time.
Hi,
I’m an HR director for a company just outside of Chicago. We are looking for a top notch QA engineer, and I’d like some recommendations for places to post to get qualified candidates with experience.
Thanks,
Marysue
I completely agree with James points here and I would add that given the rise in popularity of agile development methodologies testers and developers are moving closer together than ever before. Leaders in the software testing field need to foster this trend and encourage testers to innovate and think differently about what their new role will be. Testers will be called upon in the future to provide a more cohesive testing infrastructure that integrates with the development, build and deployment systems as well as have data that provides better insight into overall product success prior to release. None of this can happen unless we attract and retain talented people and mentor them properly.
Thankful to post. we are also writing on our blog for this.
QACampus is Center of Education for anything and everything about software testing.