Testing the Limits With James Bach – Part I

JamesBach150James Bach is synonymous with testing, and has been disrupting the industry and influencing and mentoring testers since he got his start in testing over 25 years ago at Apple. Always a great interview, James is one of our most popular guests and we’re happy to have him back for his first Testing the Limits since 2011. For more on James’ background, his body of work and his testing philosophy, you can check out his blog, website or follow him on Twitter.

In Part One of our latest talk with James, he talks about a future that involves a ‘leaner’ testing world, the state of context-driven testing outside of the United States, and why you’re “dopey” if you’re a manager using certain criteria in hiring your testers.

uTest: We know you don’t enjoy certifications when it comes to testers. In fact, in a recent blog, you mentioned that ‘The ISTQB and similar programs require your stupidity and your fear in order to survive.’ Do you feel like certifications are picking up steam when it comes to hiring and if they’re becoming even more of a pervasive issue?

JB: I don’t have any statistics to cite, but my impression from my travels is that certifications have no more steam today than they did 10 years ago. Dopey, frightened, lazy people will continue to use them in hiring, just as they have for years.

uTest: Speaking of pervasive problems, what in your opinion has changed the most – for better or for worse – in the testing industry as a whole since we talked with you last almost 3 years ago?

JB: For the better: the rise of the Let’s Test conference. That makes two solidly Context-Driven conference franchises in the world. This is related to the general rise of a spirited European Context-Driven testing community.

Nothing much else big seems to have changed in the industry, from my perspective. I and my colleagues continue to evolve our work, of course.

uTest: In a recent interview, you mentioned that you see the future of testing, in 2020 for instance, as being made up just of a small group of testing “masters” that jump into testing projects and oversee the testing getting done…by people that aren’t necessarily “testers.” Do you see QA departments going completely by the wayside in this new reality of a leaner testing world? Wouldn’t this be a threat to the industry in general?

JB: I’m not sure whether you mean QA groups, per se, or testing groups (which are often called QA). I don’t see testing groups completely going away across all the sectors of the industry, but for some sectors, maybe. For instance, it wouldn’t surprise me if Google got rid of all its “testers” and absorbed that activity into its development groups, who would then pursue it with the ruthless efficiency of bored teenagers mopping floors at McDonald’s (a company as powerful as Google can do a lot of silly things for a very long time without really suffering. Look at how stupidly HP has been managed for the last 20 years, and they are still, amazingly, in business).

Continue Reading

Are There Enough ‘Intellectual’ Software Testers?

imagesJames Bach is no stranger to tackling heated topics, and in general, being one of the most influential disruptors in the in the testing industry.

So it comes as no surprise that in a recent blog, James provided some fodder for a great discussion in the uTest Forums, arguing that there aren’t enough intellectual testers in the field — that is, testers that are willing to challenge themselves or the status quo:

“The state of the practice in testing is for testers NOT to read about their craft, NOT to study social science or know anything about the proper use of statistics or the meaning of the word ‘heuristic,’ and NOT to challenge the now 40 year stale ideas about making testing into factory work that lead directly to mass outsourcing of testing to lowest bidder instead of the most able tester.”

While there was a fair amount of pushback to this, a surprising amount of uTesters tended to agree, including one tester that even went so far as to call it a “pet peeve” of his. However, while agreeing with Bach’s assessment, these same testers argued that it isn’t necessarily their fault — it’s a product of their environment:

“To conclude, I believe that the issue lies with how projects are managed. If no time is left for more robust testing, then it almost doesn’t matter how intellectual or technically savvy a tester is if all he/she is going to have time to do is create and execute tests against specifications. In other words, intellectual testers don’t have much opportunity for more intellectual testing. A strong tester would not be able to showcase those skills in this environment.

Continue Reading

5 Ways to Learn About Software Testing at uTest

computer mouse and book, concept of online educationThe software testing world can be a complex maze, especially if you are new to the industry. There are various testing types, testing methodologies, and testing schools of thought, as well as guidance about bug reporting, project etiquette, and working on a testing team. The amount of information can be overwhelming, but we’ve outlined a few ways you can easily get your bearings and start off on the right foot in software testing here at uTest.

Read About Testing News

The Software Testing Blog is your source for news and information about the testing world. You can find posts about events, careers, trends, and specific testing types like mobile and security. The blog also features Q&A sessions with industry experts like Stephen Janaway, Craig Tomlin, and Dave Ferguson, along with upcoming interviews with leaders like James Bach.

Connect With Other Testers

The Software Testing Forums is your place to meet fellow testers from around the world and discuss the hottest topics in testing today. The forums includes over 80,000 posts in more than 5,000 topics. Take a poll, share your favorite testing quotes, or just introduce yourself to the community.

Attend An Event

The Software Testing Events calendar is a comprehensive listing of testing events happening around the globe. You can find both in-person and online events, as well as new courses available to testers. Some show organizers also offer discounts for members of the uTest Community. See event listings for more details.

Continue Reading

10 Quotes for Software Testers…Since the Last Time

So we had to dig back deep into the archives to see when the last time was that we featured some testing quotes worthy of hanging up on the ol’ refrigerator. To our horror, it was over three years ago, so we decided it was time again for another roundup. Without further ado:

Testing means learning. Learning requires faith in one’s ignorance combined with the confidence that it can be extinguished.”James Bach

“Testing is organized skepticism.”— James Bach (A double dose of Bach!)

“Be a yardstick of quality. Some people aren’t used to an environment where excellence is expected.” – Steve Jobs

“There are only two things that seem to be even close to universally true when it comes to testing – things are constantly changing, and if you put three testers in a room with a testing term or topic to discuss, no more than two of them will ever agree at the same time.”Scott Barber

“A ‘passing’ test doesn’t mean ‘no problem.’ It means no problem *observed*. This time. With these inputs. So far. On my machine.”Michael Bolton

Continue Reading

Ruby on Rails Security Vulnerability Throws Apps Off Track

150px-Ruby_on_Rails.svgOver the past several years, the web development community has been enthralled with Ruby on Rails. The combination of the Ruby language with the Rails framework has proven extremely powerful, and many of the web’s top sites are built using the two technologies. For example, sites like Twitter, 500px, Groupon and more were all built with Ruby on Rails as their framework. Both new and veteran developers have adopted the platform because of its ease of use, rich library of components, and outstanding tools.

Late last month, the gleam of Ruby on Rails dulled considerably as a new class of security attacks emerged targeting the framework. Like many security vulnerabilities, the attacks started out as academic exercises which were quickly spun into automated attack bots designed to knock over Rails servers en masse.

Today, anyone who runs a Ruby on Rails server who hasn’t applied an update is probably already compromised. Think that’s overstating things a bit? Patrick McKenzie sounds the alarm loudly in his blog post titled What The Rails Security Issue Means For Your Startup:

It is imperative that you understand that all Rails applications will eventually be targeted by this and similar attacks, and any vulnerable applications will be owned, regardless of absence of these risk factors.

Still think that’s overstating things? Continue Reading

8 Tips For Becoming a Dedicated Tester

Become a top software testerOur old friend James Bach recently fielded a question on his blog from a new tester seeking advice on what her daily routine should include so that she can grow in her new field. James seems impressed by the new tester’s discipline (she did willingly ask for daily testing “homework” after all) and dedication to the craft. He outlined five tasks he believes every tester should practice on a daily basis, here’s a quick summary of his tips:

Write every day
Whenever I find myself with a few moments, I make notes of my thoughts about testing and technical life.

Watch yourself think every day
While you are working, notice how you think. Notice where your ideas come from. Try to trace your thoughts.

Question something about how you work every day
Testers question things, of course. That’s what testing is. But too few testers questions how they work. Too few testers question why testing is the way it is.

Explain testing every day
Even if no one makes you explain your methodology, you can explain it to yourself.

I like these tips because they aren’t the typical recommendations you run across, like “test whenever you can,” “read an array of testing books” and “be open-minded when it comes to techniques.” Those are great tips too, just nothing special. Of course, James didn’t just give one sentence explanations for each of his pointers, so take a few minutes and read his complete blog post to get the full impact of these smart tips.

And as a little extra, here are a couple more tips James’ readers left in the comments section.

Continue Reading

Why Security Testing Is So Important

Voting BoothsYou can do just about anything online these, so much so that it feels like an inconvenience if you can’t complete a task online. But some things are just best left the old fashioned way.

Take, for example, the act of voting. I’m not talking about voting for American Idol (which you actually can do online now), I’m talking about voting in a major, official election. While paper absentee ballots may seem outdated, voting has proved to fragile and tamper-tempting to be shifted online. We wouldn’t know that though without some good, solid security testing.

A few years ago an e-voting system was created for Washington, D.C. and in 2010 its developers reached out to security testing experts to put the system through its paces. It failed miserably. The story is surfacing again now because the processes and results of the testing were recently officially published. The testers didn’t find some exceptionally complicated flaw only detectable with a lot of out-of-the-box thinking, they were able to completely infiltrate and manipulate the program. There’s The H with some details:

“Within 48 hours of the system going live, we had gained near complete control of the election server”, the researchers wrote in a paper that has now been released. “We successfully changed every vote and revealed almost every secret ballot.” The hack was only discovered after about two business days – and most likely only because the intruders left a visible trail on purpose. …

The security experts investigated common vulnerable points such as login fields, the virtual ballots’ content and file names, and session cookies – and found several exploitable weaknesses. Even the Linux kernel used in the project proved to have a well known vulnerability. They were also able to use the PDFs generated by the system to trick the encryption mechanism, while unsecured surveillance cameras provided additional insights into the infrastructure. While the open source nature of the code made their work somewhat easier, they believe that attackers would have been able to make quick headway even if the system had been proprietary.

Continue Reading

7 Tips for Stellar Test Management

Tips to excel at test managementWe all know what happens when you rush through testing and push a new product out to market too early (hint: crashing and burning comes to mind). In the spirit of not releasing software with major security flaws, functional defects or usability missteps Traq Software has highlighted “7 Important Principals for Test Management.” Full disclosure: Traq Software sells QA management software, but the tips are good too keep in mind non-the-less. (Numbers 4 and 5 are my favorites!)

1. Make sure you have a repeatable process. A good process helps you see where you are and where you are going.

2. Don’t cut corners prior to a release. When the delivery date is getting closer there is, naturally, a tendency to want to skip some low priority test management tasks. In doing so you hope to get the product out on time. Resist this temptation.

3. Know the metrics. Defect find rates, cases executed and lines of code changed. All these metrics help you argue the case for why the product may, or may not, be ready for release.

4. Listen to the testers. Software testers are your projects headlights. They light the road at night and help you read the map. They are trying to help you get to your destination in the shortest amount of time. Ignore their advice and you can be sure you’ll end up taking the longest route to your destination.

5. Employ a good QA manager. The QA manager is like the pilot of the plane guiding the product to touch down. … good QA manager has an eye for balancing the demands of time, quality and features. He or she is worth listening too carefully.

6. Get the customer involved. The sooner your customer starts giving you feedback the sooner you can correct issues.

Continue Reading

Announcing the 2011 uTester of the Year Awards

2011 uTester of the Year AwardsToday, we’re thrilled to announce the results of our third annual uTester of the Year Awards. Every year, we recognize uTesters who have consistently gone above and beyond their call of duty in their participation with uTest projects. This year’s winners were selected by our community and project management teams, who have had the privilege of working closely with such an extremely talented community of professional testers. From test automation to test team lead, these winners are truly experts in a variety of testing domains. The level of talent continues to impress, with each year’s accolades becoming more and more difficult to attain (and judge). So without further ado, let’s meet our 2011 winners!

Top honor for the 2011 award goes to David Honeyball from the United Kingdom!

David joined uTest in June of 2009. Since joining us, David has become a Gold rated tester in Functional, Load and Localization testing, as well as having achieved the silver rating in both Usability and Security testing. David also became our Top Test Team Lead in 2011, successfully leading nearly 200 projects alongside of uTest project managers. David had this to say about his experience with uTest during the past year:

I joined uTest back in the summer of 2009 and have to say I have never looked back. What started out as something extra in my spare time has taken up more and more of my time in a good way and has increased my confidence as a tester to levels I never thought possible. I have been a tester for nearly 15 years but can safely say that every day with uTest is a new experience and a new challenge.

I have met so many wonderful people including other testers, CMs, PMs and customers who are all committed to achieving their goals and creating a wonderful service. They have guided me and helped me in the last year and increased my communication skills as well.

TTL (Test Team Lead) Experience:

2011 has been a big success for me personally, as it has for uTest in regards to growth and development, in many ways due to my journey from tester to TTL. I started out as a TTL early in the year and felt at the time that it was a great way forward and would be of huge assistance and help to customers. Since then I have been involved in something nearing 100 cycles as TTL, but that could be more now!  I enjoy the TTL role immensely as I get to speak to the PMs and testers and help out others who are stuck. I hope if you have worked with me that you know that I take that side of things very seriously. One of the great rewards is helping someone who is stuck to complete a test case or test for example. Above all, I believe that with my experience I am a very fair TTL and have the best interests of customer and testers close at heart.

As time goes on I hope to grow more and more into this role and improve further as there is still so much to learn.

Special Projects 

Apart from testing and TTL work which does take up some time as you can imagine, I also help with test case writing for certain PMs which I enjoy and get value from. I was also heavily involved in the startup of the test case conversion to uTest which is the new system we see today. This side of things linked in with my testing and other roles forms a vital component of what I do as well and hope it adds value to the company as a whole.

So, just want to say a huge thanks to everyone involved with uTest for the opportunity and long may we continue to grow together and become invaluable to customers across the globe…

The complete list of winners is shown below:

Continue Reading

Software Engineering Hits High School

Software Engineering Hits High SchoolA teacher in Massachusetts dedicated a computer class to developing and testing mobile apps. The Education Secretary in the UK is calling for a total program overhaul of country’s computer education curriculum. Now, the Mayor Michael Bloomberg of New York City has declared that an entire public high school will be devoted to teaching students software engineering. From Government Computer News:

“Today, far too many of our graduates are leaving without the skills they need to succeed beyond high school. Not every student wants to go to college, nor is college right for everyone. But all students should leave prepared to succeed in the next phase of their lives,” Bloomberg said. “It’s a new way of thinking about secondary school based on today’s economic realities.” …

Frank Thomas, a spokesman for the city’s Department of Education, anticipates that the school will have between 420 and 460 students by 2015, when all four grade levels are enrolled, Adrianne Jeffries reported in BetaBeat. The school will start with a ninth-grade class this year and add on another grade level for the next three years.

The city has other specialized high schools for science, math, the performing arts and other subjects, but it did not have one focused on computer science. …

Joel Spolsky, a board member of the new school, said one reason he’s a proponent of the school is that it could can train many excellent software engineers who are not currently at the top of their class academically.

“I think this is the best thing about the school,” he said in a blog post. “A lot of kids are just not interested enough in other academic subjects to get good grades, but they would make great software engineers. A lot of immigrants (especially in New York) are not yet proficient enough in English to get good grades in all their subjects, but they’re going to make great software engineers, too.”

I have to say, one instance is cool. Two instances make you raise an eyebrow. Three instances (especially when they’re consistently bigger examples) might just be the start of a trend. And this trend of focusing not only on computer basics, but on more advanced – more engaging – computer topics that can lead to lucrative, fulfilling career paths is long over due.