Testing the Limits with James Bach – Part I

Back by popular demand, James Bach has returned for another Testing the Limits interview. An author, speaker, coach and consultant, James is one of the testing industry’s most compelling figures – constantly challenging conventional wisdom with style that is his alone. If you’re new to James Bach, you should – nay, must – spend a day or two on his blog.

In part I of our two-part interview, James discusses “pathetic compliance” and other newly coined terms; weeding out bad ideas in the field of testing; the increasing popularity of context driven testing; Apple’s antenna bugs and more. Be sure to check back tomorrow for part II.


uTest: When we interviewed you last year around this time (here and here), you had just published Secrets of a Buccaneer Scholar. What have you been up to since then? Any new books in the works? New speaking opportunities? New theories? New hobbies & interests?  Said differently, what’s coming in the next 12-18 months for the Buccaneer Tester?

JB: I coined the terms “test strategy playbook”, “pathetic compliance” and “thread-based test management.” I’m working on a book with Anne-Marie Charrett on how to train testers over instant messaging, and done a lot of online coaching along the way (I offer this for free on an as available basis, or if I’m paid I’ll schedule a lesson). I joined the board of the Association for Software Testing, again.

The main thing I’ve been doing is being test architect for a class III medical device. I got to read lots of standards and regulations, design a statistically justifiable accuracy testing protocol, create a log file analyzer with automatic graphing, document a mathematical algorithm, and I developed a tool for reshuffling pages in dozens of PDFs to create topical doc repositories.

uTest: Your domain name is satisfice.com – what does the “Satisfice” name mean to you, and why did you choose this name?

JB: Satisfice is a word coined by Herbert Simon, who won the Nobel prize in Economics (technically, it’s the Nobel memorial prize or something like that) for his work on bounded rationality. Satisfice is a verb meaning “to achieve a result that is good enough.”

Much of what I do and say is based on the work of Herbert Simon. He’s one of my heroes. Also, the domain name was available…

uTest: Is the testing craft better or worse off since that time (Nov, 2009)? Which developments or trends point us in either direction?

JB: I don’t know if the craft has changed in any significant way, but my business has picked up a lot. It must be the economy getting a bit better. By the end of the year I will have taught in ten countries in the last 12 months.

uTest: You recently said that, “There is no such thing as an ‘exploratory tester’ except inasmuch as a good tester obviously can and will do exploration as a basic part of his work.” Are there too many “labels” in testing? If so, what are the consequences?

JB: I would say we need more and better labels, while retiring some old ones that don’t help. The consequences I’m worried about are what happens when people say they are tired of this profusion of terminology. That’s like saying “I’m tired of software companies creating new technology! Let’s go back to HTML 1.0!” That’s like saying “I’m tired of learning. It hurts my brain. Everyone has to think the same thoughts now!”

I’m sorry that testing is complicated, folks. No wait. I’m not sorry at all. Go away if you don’t like it. I think we will be creating new ideas and labels at a good clip for years to come. Let’s do a good job of it, though, and weed out the silly ones. (For instance, no one should utter the term “boundary value analysis” ever again, unless they are talking about actual analysis, rather than the simple act of adding 1 and subtracting 1, which is a form of “analysis” any six-year old can routinely do.)

uTest: The amount of testing-related content on the web is exploding. At the same time, the Context-Driven school of Testing has never been more popular. Coincidence?

JB: I wouldn’t be surprised if CDT has never been more popular, because we produce lots of new ideas, actively weed out the ones that don’t work so well, and we share them freely. Go to any practitioner’s conference and you’ll either see the Context-Driven community frequently referenced, or else you’re at an ISTQB/ISEB Zombie Rally, (or whatever they call their dark gatherings by moonlight when they invoke whatever evil spirits and marketing consultants that help them deceive the unwary.)

Also, the Association for Software Testing has officially declared itself a Context-Driven Testing organization, and the first conference dedicated to Context-Driven Testing will be held next July in Seattle (the 2011 CAST conference, which will be run by my brother and I).

Context-Driven Testing represents the free world of testing. We’re sharper, braver, hipper, and we believe in working at our skills instead of smugly assuming that we are already skilled enough, so of course it should be popular… Except…

Except three things:

  1. I don’t actually know that it is more popular then it ever has been.
  2. If it isn’t more popular, there’s only one other possibility: that’s it in decline. That would make me sad, so I hope that’s not true.
  3. The second point was such a downer I can’t think what else to add to this list, but there’s probably other caveats.

uTest: The divide between hardware and software seems to be narrowing. We’re thinking specifically about the problems encountered by Apple (antenna) and Toyota (brakes). If true, how does this impact the craft of testing? Is there a role for software testers on the hardware side of things, and vice-versa?

JB: Oh yes. The Way of the Tester applies to anything.

(Caution: Don’t apply it to your wife and kids, unless you like grumpy housemates.)

It’s all just rapid learning, analysis, questioning, the relentless exploration in search of order and disorder. I’m not an electrical engineer, but I had to learn about electronics for my recent project. Give or take an Ohm’s law and a phase angle or two, it’s all the same stuff. Testing is testing.

uTest: Along those same lines, serious bugs have proven to be as much a PR problem as a development problem. Any advice for companies that must deal with their bugs in the public spotlight?

JB: Yes! Come clean and stop complaining, Steve Jobs! Of course your products have problems. And once in a while they are serious. Instead of whinging about it, have your PR firm get the test manager out front and having him talk about the challenges of producing excellent products and how the best companies are distinguished by how they learn, adapt and respond, not by how they refuse to take risks or admit mistakes.

You can make people trust you not just be never making a mistake, but also be showing that you can make good on your service despite mistakes.

Editor’s note: Read part II of the interview.

Essential Guide to Mobile App Testing


  1. says

    First of all, I don’t have a website so posted my LinkedIn profile link.

    I am not sure which experience “Anonymous Bullies” (read Bryan, Gary etc) are referring to? In my 5 years of work exp as a tester, exp has been gradual not repetitive….I saw many failures of so called processes and all “bullshit”. I do hold a certification for last 3 years which is going to expire very soon and I don’t see any reason to renew or upgrade it because I applied my practical learning and could convince stakeholders as there are better ways to do things in testing. I had to work hard, had to read a lot more, had to look at more Open Source projects, for months to change very small things like metrices, bug/defect status report etc and thankfully they were all ears. I am not an expert but I do call Bach brothers, Michael Bolton, Dr. Cem Kaner etc “EXPERTS” because their way of working make sense to me, CMMi ITIL or any other processes are good but these processes are not implemented the way their documentation shows the reason I see is “Practical Limitation/Ignorance”, there is no independence to apply brain/logic/heuristics.

    So “Anonymous Bullies” stop being anonymous, come out in Public, I promise, You won’t need a James Bach to prove a thing, any of their self acclaimed students (like me) can argue and win over you.

    All the best for coming out in public.

  2. says

    I haven’t commented before now simply because I don’t see that James, Jon, Cem, Scott, or anyone else listed here needs my help or support in defending themselves from somewhat anonymous internet challenges to a body of work they’ve been developing over decades.

    I’d simply like to agree with Judy that the “pro from Dover” syndrome prevalent in modern business is a pernicious myth we all need to debunk as soon as possible. No outsider, no matter how famous or charismatic or expensive, can effect change within your company just as no negotiator or diplomat can get two countries to stop warring and start playing nice with the trade and cultural exchanges.

    The desire for change has to come from within your company and it must be shared by a few people. Don’t just take my word for it. Read any book from a successful business person about their experiences managing their companies in any industry. They all agree that change is necessary to stay competitive in a shifting marketplace. They also agree most people at their companies did not want to change when it came down to actual work, even if everyone agreed change was required to stay in business. Change takes a “change agent” who can champion the changes necessary and sell, charm, bully, or threaten others into doing what needs to be done. Its most effective when that’s an internal agent, but it can be an external agent who is not vested in the internal politics of the company.

    If the company is not vested in change, then what Judy said happens will inevitably happen. That bastion of free thinking hugs Jack Welch talked about it as well. If the rank and file is not brought around to the change, nothing will change. All the work and new processes will die a death of a thousand cuts as the old processes are rationalized, justified, and slowly brought back online.

  3. says

    At first I thought I would not reply to the strange and fact-free comments from Bryan, Gary, etc.

    But then I thought, maybe they need a reminder, so here it is:


    I’ve been testing, teaching testing, and publishing new ideas on testing for a solid 20 years.

    My email address is published. I’m on Twitter and Skype. You can even call me on the phone. I show up at many conferences and I will argue with anyone and demonstrate my work and ideas in action to anyone. If you haven’t approached me to discuss and debate testing, what more can I do to encourage you?

    Do you want to build a better craft? Then stop skulking. Use your full name! Stake your reputation! Stand up for what you apparently believe and defend it before an audience of your peers. This is what Cem Kaner, Michael Bolton, and I do. This is the context-driven way.

    We. Don’t. Hide.

  4. says

    It’s interesting to see from the anti-bach sentiments that it is clear to me that the *attackers* are unable to distinguish fact from personal attacks. I have discussions with Rex Black and the Bach brothers and with Cem Kaner and many other people whom I consider to be leaders in our craft. I learn from all of them. Just because someone has been testing for 14 long years doesn’t mean that they are worth listening to – BUT they may have an idea or two worth considering. Context driven IS the way to go because that is WHAT we practise all of the time in life (to varying degrees). AND context is what we practise in testing regardless of the approach. The exploratory approach is ONE way to test just as scripted robot testing is another. It is the degree of exploring that varys.
    It just so happens that James has decided to tell what he see’s as the state of testing thereby prompting discussion and/or debate? Isn’t that a good thing?
    However I have NOT seen ONE rational point in the anti- comments above other than the feeling “James stop challenging my own little world – i don’t want to think!’
    Testing is something that we do naturally as humans -testers have evolved the thought processes a little further. Why not take that thinking and extend it? Why paint ourselves in a box? Learning occurs in many forms.
    AND james has helped me alot, as has Michael Bolton, Scott Barber, Cem Kaner, Doug Hoffman – FREE – all with the purpose of helping me become better. Isn’t that our goal after all – to become better?

    Michael Bolton makes a good point – The anti commentators – where are you? Why not open yourself to debate or is it enough for you to comment and run without substance behing *your* claims.

    Twitter ID:bjosman

  5. Mohinder Khosla says

    Michael is right in saying that testers have more in common with editors or film critics than with widget-checkers. I would go a step further and say that testers are information providers through their hard work in exploring the SUT and reporting any anomalies, non-conformance or ambiguities with the products which testers come to call them as defects, bugs or errors. If you talk to any reporter, he/she can tell you they will use any available source or tool at their disposal to get to the truth before making it public. Testing is no different and if we are passionate about testing and majority of us are then we should adapt to all new ideas irrespective of the source. I do not use all that is coming from Exploratory testing school of thoughts and train my brain that works for me but I do follow great minds such as James that helps me formulate my thinking.
    Being a Science graduate, I do sometimes describe exploratory testing like experimental science, this is me thinking that sciences of Exploratory Testing is not mature enough to be comprehensively written to books.
    Good luck to those who disagree with James and rest of us having a deal of respect for him. Let’s not forget knowledge when shared freely makes the world a better place and brings harmony among people.

  6. says

    Just to clarify, the previous John comment was me. My website is under construction and I didn’t really want to toot my own horn, but I also don’t want to be anonymous either.

    To Judy, the trenches is where I live. I do not enjoy coming in to save the day as a traveling consultant. My preference is to stay somewhere and build something. It is tough when you are not a charismatic personality but it is worth it.

  7. says

    James Bach spent almost a day working with me on testing problems for free with no mention of the ASQ, his books or his course. Why would he do this for free? Because he enjoy’s teaching and enjoys testing. I believe he is hoping to make the craft more enjoyable, productive and fun for everyone. I believe some of the comments in previous posts are uncalled for. I have no problem with questioning methodologies but I believe it can be taken too far when questioning the man.

  8. says

    Who’d have thought that the comments to this interview are more hilarious than James’ replies to the questions? (Sorry James).

    I’d like to pick up something that Judy said:
    “Having been on both sides of this coin I think that what consultants live in a testing reality is very different from what the troops in the permanent trenches experience.”

    I agree, but the reasons listed don’t sound right to me. Yes, it’s easier for someone external come in and recommend a change, managers are more willing to listen. But ONLY relying on that to say people in the trenches can’t change anything is an excuse in my opinion.

    I started using ET after finishing Michael’s course. First I only did a bit after the scripted testing was done. Then, on selected projects (the ones where you have 6 browsers to test in one day, never seen the application before, yes, done it and it works), which showed that we can discover the worst bugs before the application ships without having written a test script. After doing that once, did the second project, said to the PM, speak to that guy over there and see what he says about how I tested. He did, came back and said I’d like you to do that on my project as well.
    After seeing that I did a lot more /testing/ rather than /documenting/ they wanted me to use the ET approach. I haven’t written a single line of script since.

    We’ll now use this approach on a bigger project with external test consultants coming in. We’ll document our tests in Word while carrying out the tests, use a tool to record a video for each session and have debrief afterwards. I’m sure that I’ll have more confidence in what state the product under test is at any given time than when I looked at how many scripts had been completed.

    To come back to my original point, if you start slow there’s no excuse to not implement ET if you are working in permanent employment.

    The argument that you need skilled testers is true, to an extend. But hey, as the critics say, ET only uses fancy new words for what is essentially common sense. So you WILL have to use your brain and you WILL have to learn new things, some of which invalidates what you learned before. I see this as an opportunity, not as a problem. IF a tester is willing (note the capitals) learning ET is not the issue. Sitting in the trenches of ones own mind and saying “But that’s now how we’ve always done it and I feel uncomfortable with that.” is a problem and won’t get you further. I’ve seen people complete the ISEB Test Manager course that I wouldn’t trust to give a decent size project to manage. Knowledge comes from the willingness to learn something new (and to get out of your comfort zone), not from a course.

    Recommendation to use ET was by word of mouth within the company. Once people see that you’re doing something useful they want a piece of it if it makes their life easier as well. And that’s exactly what I’m doing.

    Once that big project is over I’ll report back, watch this space (You’ll find it, I’m sure).


  9. says

    Now this is an interesting phenomenon: Bryan, Gary, Martin, and Judy all post without last names and without links to their identities, where everyone else (with the exception of John McConda, who outed himself on Twitter) provides a last name and a link.

    Bryan accuses James of marketing because James points out the context-driven-testing Web site and Cem Kaner’s Web site, both of which are decidely non-commercial. If James had really wanted to market himself or his cronies, wouldn’t he have put a link to his own Web site, or to, say, mine? (Note to self: persuade James to do that more often; we’re missing all kinds of marketing opportunities.)

    The idea that there’s no knowledge base for context-driven testing to be built on is silly. There’s a book (Lessons Learned in Software Testing) and another book (Testing Computer Software), the Black Box Software Testing course (available entirely free online, and in a facilitated version offered by the Association for Software Testing to its members). There are tons of Web sites on which foundational context-driven material is posted, again for free, at Cem Kaner’s web site, at James’, at my own, at Mike Kelly’s, at Karen Johnson’s, at Scott Barber’s, and at the sites and blogs of so many of our colleagues. More importantly, the knowledge base for context driven testing is out there in the world, in Jerry Weinberg’s work, in Herbert Simon’s work, in Nassim Taleb’s work, in Malcolm Gladwell’s work, in J.B. Rainsberger’s work, in Joel Spolsky’s work, in Edward Tufte’s work, and yes, in Boris Beizer’s work—all kinds of stuff related to software development and testing AND economics, anthropology, psychology, statistics, medicine, mathematics, philosophy… The mandate for learning about testing is interdisciplinary and global. Alas, it can’t be covered by a 40-question multiple choice test of highly suspect validity.

    The end of Bryan’s remarks tail off into incoherence and inaccuracy (we argue that testers should be skillful, highly trained; not unskilled, notcommodities), so let’s move on.

    Gary raises an interesting point about “getting wealthy”. I observe that an ISTQB certification exam costs a minimum of $100 (in India), rising to $250 (in the U.S.). There are 130,000 ISTQB certified testers worldwide. That is at a minimum $13,000,000. Where has that money gone? Has it gone into the development of free online courses? Published material for testers who wish to engage in self-education? A not-for-profit association focusing on developing tester skill? Not-for-profit conferences? I’m afraid that ISTQB-certified testers are not zombies; rather, that they’re in general well-meaning, hard-working human beings whose livelihood is threatened by ISTQB marketing, and whose pockets are being systematically picked.

    Judy errs in suggesting that exploratory testing isn’t documented, when in fact exploratory testing can be highly documented. This mistake is understandable if you follow the ISTQB’s (or TMAP’s) descriptions of exploratory testing, instead of accounts from people who actually study and practice it. (I’ll have more to say about this on my blog fairly shortly. To avoid the charge of self-promotion, I’ll encourage people to find the link themselves.)

    I understand the difference between quality control and quality assurance, which is why I suggest that testers in fact do neither. These terms come from the world of manufacturing, where it’s important to make each copy of each widget look like the master widget. Software (except at the stage that it is being mass-produced and duplicated onto CDs) isn’t much like that. That’s why I think we need to focus away from the manufacturing model, and more towards the design or publishing model. Testers have more in common with editors or film critics than with widget-checkers. Our role is not to decide whether something is bad or good, but rather to assist our clients, through active and engaged investigation, in determining whether they have the product that they want.

    The proficiency argument—that we can’t expect testers to be skilled— is particularly sad. There seems to be an implicit assumption that testers can’t be mandated to have the qualifications, the freedom, and the responsibility to do excellent work. Yet you don’t hear airlines saying, “We’d love to hire skilled pilots, but they’re so darned expensive and hard to find. Or we’d have to train them, and keep training them.” There’s a feedback loop: if you view testing as an unskilled occupation and the value of testing work as low, you’ll get less value from it; and if you get less value from it, you’ll be more inclined to view it as work for the unskilled, further reducing its value until it fades into irrelevance. That’s not a problem that we’re going to solve by shrugging our shoulders and commoditizing testers.

    I disagree with Judy’s inference that we don’t understand what the “troops in the permanent trenches” experience. We’ve done testing in a wide variety of circumstances and contexts. Based on that experience, we take what we’ve learned and try actively to help people develop skills to get out of the trench. We try (and largely succeed, in my experience) in re-engaging testers with their skills, their passion, and their craft, by recognizing (among other things) that testing is more than checking, that reporting is more than filling out forms, that finding problems is more than comparing the product against a specification—and in doing those things in a way that genuinely matters to their clients. One motto comes from Richard Bach: “be so good they can’t ignore you“. We encourage testers to learn how to do that—and to help each other, including us, to do that.

    —Michael B.

  10. says

    Well its true that is hard if you are not ¨expensive and famous¨ to make changes.
    But the gain is in the passion for the work, where you can have a platform to use it in other activities: puzzles,way of thinking.
    Its not good to be scared too much about following procedures and approaches. If you become a robot you will have bigger chances to be outsourced for a cheaper robot so you will loose your job much faster anyway.
    Do not forget: coding is always easier to outsource than complex thinking; following step by step procedures is also much faster to outsource.
    What is not complex will be sooner or later nullified almost because others can do it. If more people can do the same thing as you, your value will drop.

  11. Judy says

    I have to agree with Bryan and Gary. I wrote an e-article for STP titled “Let’s embrace the right practices” expanding on the idea that Bryan is puttin forth here and one of the people responding misunderstood it as already having been covered as a part of Cem and James work. Not so. I also agree with Gary that James and Jon posit ET as the be all end all when it isn’t. And it’s not just for political reasons of mandatory documentation in certain contexts but also practical reasons like the level of experience needed to be proficient at it, the will of your company to allow you to move test support for their apps to a lightweight model when the reality of your company is that they barely understand the difference between QA and QC and what they think they want vs. what they actually need and the list goes on…. In reading this interview I wondered if it occured to Jim that at least part of what he’s geting a pass on from his client is that he is the new guy. I wondered if Jim spoke with the “old guy” to find out what the political climate of the company that resulted in him doing what he was doing. I don’t bring that to the table to excuse the behavior but I’m not naive and Jim’s recount seems just a little cavalier and project hero-ish. Failing to admit that our testing profession is being pushed hard by a lack of standards and a host of other ills smacks of elitist outsiderism. Having been on both sides of this coin I think that what consultants live in a testing reality is very different from what the troops in the permanent trenches experience. And much of the so called advice is worthless because it doesn’t help those people change any hearts and minds. Often famous consultants can and sometimes do change activities at a company, at least for a brief time, but it isn’t because they are saying something the leadership hasn’t heard before; it’s mostly because they are expensive and famous and the leadership thinks they have to listen to (at least for a time) because they paid for the ticket. After a while they go back to what they were doing because they never do anything about their cultures…. I like James passion but I must admit that I don’t think most people can pull off his act. Not because he is so great, but because they aren’t famous and expensive and outsiders….

  12. says


    I don´t think that if someone has worked at Microsoft that means a lot. Big companies are known, with few exceptions, to treat the employees low because they have many of them.
    Notice that Microsoft products are notorious in breaking a lot of the times. (That´s why some people hate them and use alternatives) I think ET and just simple basic checks should be needed in reducing this.
    I think ET is the best approach and the rest are just extensions of ET. When you do a script then this script would have value after if it had fits in ET analysis.
    ET and deep study of it is needed because:

    1) As a tester I am not in the mood to adopt too much coding to please those who don´t respect testing unless is hierarchical a sub-developer position.
    And making more code is getting closer to a developer´s value.
    Its about respect first of all, to avoid being monkeys.

    2) If you have a look-around only to an application with the purpose to give feedback about it, it doesn´t mean that is not valuable work. On this principle then all the management in the world should be replaced with small scripts, because managers don´t do anything according to this.

    3)Gary, you don´t look a person with much personality anyway. You accepted bad general views about testing and now you look to contribute to make a tester´s job more sh***er. I don´t like too much people who want to make my job as a tester one with lack of personality, monkey-type and that contains fake-values.


  13. says


    I thought that I should add my penn’orth to this discussion as I have experience both of the ISEB/ISTQB “way” and the SBTM/ET approach advocated by James and Jon Bach, Michael Bolton, et al.

    It is true that in some circumstances the highly scripted approaches advocated by the ISEB and ISTQB syllabus are appropriate. For example a safety critical application needs to have every ‘i’ dotted and every ‘t’ crossed. In such circumstances the highly detailed planning phase is gone through and requirements don’t really change much. It is therefore possible to go through the SDLC flow in much the way ISEB/ISTQB describe. In my experience – both personal and from talking with others – such projects are few and far between and are limited to very special organisations.

    Back in the world most of us inhabit things go wrong in projects, development time eats a way into the testing schedules, pressure gets applied on testing because “the end is in sight” to do “what we can”. We need to have a way in which we can break the deadlock whereby testing is seen as a bottleneck and this requires rapid revisions of our testing strategies and plans.

    When we are having conversations with business managers, project managers, customers and other stakeholders it is much better if we can say “yes, I can test this application for a day. I’m going to be limited in what information I can give you so what would you like me to concentrate on?” rather than “I have not got enough time to run all my tests. A day is too short a time because my test plan says I will take 5 days to test this area, whinge, whinge, whinge…”.

    The SBTM approach is something that I have found very helpful in building good working relationships with my stakeholders. I take account of the contexts in which I and the rest of the project team am working and have a powerful negotiating tool at my finger tips. It also means that I can get on with testing.

    I do not believe there is a single be-all and end-all approach to testing and nor do I think James, Jon, Michael, et al consider there to be a single approach which works. It depends on the context.


  14. says

    Wow, I’m surprised no one called us “ET thugs” this time! We must be making progress.

    SBTM is a method that’s not for sale, not trademarked, not patented. The cost, however, is that you have to be smart enough to see the value of it in certain contexts. Apparently having that kind of brain power is too high a price for our critics to pay.

    The tool we created is open source and free. We’ve written no book on it. In fact, people are pestering us (politely) to write one, or any book on ET — but no, I guess we’re too busy making money from all of our … *books on the subject*?!? Wha???

    When James and I (and several others) describe exploratory (or rapid) testing, it’s as an approach that can help testers design, execute and report the results of testing they’re expected to do quickly and under pressure with little background and preparation, yet thrive despite being under high scrutiny.

    Our critics have never been in that situation, I suppose. That’s ok, it just means they are disqualified from being able to understand any benefit from our tactics and techniques.

    As guru, I command you, dear Zombie-Disciples, to study our critics well, for they have managed to acquire all the time, budget, and resources they need to do testing well. Study their test cases for they are the blueprints for all the skill they will ever need.

    I am guilty of wanting to help testers win more credbility and respect. But maybe our critics are right and “playing around” is all the language we need, because this ET stuff is common sense.

    I’ll happily let our critics describe their testing through test cases they executed. While they count the ones that passed or failed, we’ll be relegated to using our brains to investigate the ones they missed. Oh, well.

    While they follow what their ISTQB proctor told them, we’ll slog it out in the ambiguous world and continue to make up words that have no value to them. We’ll do crazy things like use the word “conjecture” to describe a theory; “refute” to describe what we do to an assumption; talk about “bias” to convey our perspectives; “branch” to describe a course of action on a thread; invent “heuristics” to describe our half-baked solutions, and make sure we have sufficient “oracles” to know how we’re doing.

    But maybe as we do that, the executives who tend not to see testing as a serious science will ship their jobs writing and executing test cases to the lowest bidder, making more room for us to get better at our craft and establishing our reputations as critical thinkers.

    Listen, critics, I love ya’. You reacquaint me with my zeal for ET. But if you’re already empowered, suggest what you did to get that way. Else, get *out* of the way.

    Meanwhile, James and I will focus on talking about epistemology for the rest of us.

  15. says

    I can only offer up my personal point of view, so take it for what it’s worth.

    My one in-person encounter with James was a healthy debate about the role of taxonomy in testing. We didn’t necessarily agree, but it was a vigorous, earnest and educational discussion. And I agree with the earlier commenter who said that they’ve found him open to other points of view. In my book, that’s the way the larger discussion gets advanced — earnest, open-minded debate.

    Also, a few members of the uTest team attended STP Con 2009 in Boston and saw James present. Again, they didn’t mindlessly agree with everything he said, but their feedback was that he was thought-provoking and they learned a great deal.

  16. says

    Martin –
    As you said – “The Bach approach is all about selling books and courses and pontificating to the great unwashed. No thanks.”

    I have some talks with James Bach on skype. He is always ready to help & teach testers for FREE. I remember, couple of times he spend hours on me to teach some basics of testing.

  17. John says

    James’ style may be tough to take sometimes, but in my experience I’ve found him open to other points of view, so long as they are your own and you can defend them.

    I think his message is the opposite of brainwashing. The whole point of Context-Driven is to tailor your testing to your own situation. The opposite of that is to read a book or take a test and then try to apply that as a set of “best practices”. That sounds more like brainwashing than anything I’ve heard from the CDT camp.

  18. Martin says

    I have heard James and his brother speak a number of times but I am yet to walk out of the room feeling anything other than that I have:

    1. Been told a bunch of things about testing that everybody already knew (but with new buzzwords to replace the old ones)
    2. Been brainwashed (attempted at least) to believe that anything other than Context driven testing automatically meant I was lazy and uninterested in progressing my craft.

    I agree that a foundation level certificate from ISTQB/ISEB is fairly meaningless paper (that has value only as the basis for further study and experience) but I do NOT agree that the associations themselves are interested in creating Zombie testers. Their goal is to unite testers through a common language – for no other reason than to make the profession more professional.

    The Bach approach is all about selling books and courses and pontificating to the great unwashed. No thanks.

  19. Gary says

    I have read and listened to both James Back and Jon Bach in my long 14+ year tenor of test engineering. Since then, I have come to the conclusion that exploratory testing has its values in particular realms it is NOT the end all, but it’s not the ‘end all, be all’ of test methologies as they make it out to be. I have also met and have extensive conversations with Rex Black. Rex and I agree on one thing that James and Jon Bach are preaching to people that exploratory testing is the end all, be all answer to testing when it is not.

    People..people, think for yourself and don’t let these guys (the Bach brothers) use their unjustified (unearned) glory for being so-called ‘testing gurus’ or ‘testing visionaries’ to make you think that ‘Exploratory Testing’ is a the best way to test a product or software when it’s clearly not.

    In most reputable software or testing companies even ‘Session-based’ Exploratory testing is not worthwhile as most senior level management do not see value in this form of testing. This definitelly holds true in the largest of large software companies such as Microsoft. (I worked there before)

    I think it’s a shame that people like James Bach are getting wealthy off of promoting their so-called ‘end all, be all’ solution to software testing methodology when it’s not anything even close to as they make it out to be.


  20. Bryan says

    Hmmm … ouch – plug for James’ own school of thought (http://www.context-driven-testing.com – single web page?); plus a plug for his comrade (Cem Kaner – Association for Software Testing Director & Executive VP); plus a plug for his own Conference …looks as the “evil spirits and marketing consultants” have possessed more than ISTQB.

    The only thing I can definitely agree with is that testing is a context based process. However, so is development, life and business processes – and they have foundation knowledge bases to build upon. It is a shame that there is no testing foundation knowledge base for context driven testing to build on … oh wait … isn’t that ISTQB … better not have standards then … after all – anyone can test – no training required!


Leave a Reply

Your email address will not be published. Required fields are marked *