Software Testing Blog

Our guest blogger this month is Travis Howell. A member of the uTest community, Travis has over 15 years of project management/analyst experience, including one year at NASA where he served as Deputy Manager of Human Spaceflight for the International Space Station. A father of three (or seven if you include dogs), Travis is an employee by day, a father by night and a uTester by early morning. Sleep would be nice, he said, but he’s adjusted to living on a couple hours.

In this post, Travis (a Gold Tester) outlines the mindset needed for success in the uTest community – including advice on logging bugs, preparation, knowing your limitations and more. Enjoy!

*************

How fitting to the time of year. It’s summer, it’s hot – for some of us it’s very hot – and a pool looks so inviting. Tell me this: Would you jump into a pool if you didn’t first know how to swim?  My guess is that your answer is NO. This world presents a lot of open invitations – invitations that appeal to our wants and desires and make it easy for us to just jump right in. I have to admit, sometimes it’s nice to be first. After all, we live in a generation that rewards the person that comes in first. 

Testing Is NOT About Being First
It’s not about the first person to find bugs or the first person to complete their work. To test is: to be informed, to understand, to question, to investigate, to identify, and to communicate. In the end, it’s about providing the customer with the ability to deliver a superior product. And once they bring that product to market, who is ultimately the winner?  You, the consumer.

What’s This — A New Test Cycle In My Inbox?
A new test cycle is an open invitation for everyone to jump in the pool. I know how inviting that test cycle invitation is, and I can guess what’s going through your mind. “What do I have here, a new test cycle. Looks like I’m the first participant, the world is my oyster. I’ll get in, look around, get a jump on everyone, score a few bugs, rack up some dollars, and out I go.”  Easy money right?  Maybe, maybe not.

Let me take you back to my first question, would you jump into a pool if you didn’t first know how to swim? Now, let me re-phrase it as it relates to testing. Would you enter a new test cycle without first knowing what the expectation of the customer is and what is expected of you to test? The correct answer is NO, but the lure of ‘easy money’ is there and provides the temptation to push a few in the pool. As an example, I have seen a large number of bugs logged to a test cycle where it is clearly identified within the objectives that testing is not to begin until a future date. We’re all guilty of moving too quickly sometimes. But don’t forget one thing, what you don’t know could come back to bite you.

Read more…

Software testing doesn’t have to suck, says Ben Kelly, the latest contributor to our guest blogger series. A resident of Tokyo, Japan, Ben has over seven years of software testing experience – including his time as a member of the uTest community. In his spare time, he practices Kendo and has represented Australia twice at the World Kendo Championships. He was a Jedi knight for approximately 1.5 seconds and sometimes writes about himself in the third person.

For more of Ben’s writings, you should go read his blog or follow him on Twitter.

***

A good friend and fellow tester, Jared Quinert said to me recently, “It’s important for testers to know their job doesn’t have to suck.” I’d never really looked at it from that angle before, but really I think it’s an important point to make. I think many people are sold a story about how testing is boring, repetitive drudgery that requires little skill. It’s bad enough that many of the people we interact with – programmers, project managers, BA’s – tend to believe this. The really sad thing is this line seems to be bought by so many people that would call themselves testers.

Receive spec, create traceability matrix, write scripts, execute scripts. Repeat. Apologize for not finding all the bugs. Promise to do better. Repeat. Orwell himself could not have written a bleaker, more dystopian view of the future.

No thanks.

I want to work in a place where the developers are up in arms if a project manager suggests shaving time off testing to meet a deadline. I want to work in a place where the BA’s demand that the testers help analyze risk during design. I want to work in a place where project managers and stakeholders listen intently when a tester says ‘I think there might be a problem here’…

Here’s the thing though – it’s up to you and me to build that environment. It takes constant effort. It’s hard work. It’s challenging and sometimes it’s disheartening. You’ll make mistakes. People will misunderstand you, doubt you, sometimes oppose you. If you have a mind to try though, it can be one of the most rewarding things you ever do. Having a developer say to you ‘wow, thanks man. If that had gone to production, it would have been my neck on the block’ – that’s a massive win right there.

Read more…

Santhosh Tuppad has high hopes for the software testing industry – and he’s here to tell you all about them as this month’s featured guest blogger. Over the last year or so, Santhosh has proven to be one of the top testers in our global community: He is an active member of the uTest forums, Bug Battles and, of course, numerous customer test cycles. His rise through the ranks of the uTest community was highlighted as part of our Tester Spotlight series.

Aside from uTest, Santhosh is a member of the Weekend Testing, and received his formal training from fellow guest blogger Pradeep Soundararajan. For more on his background, go check out his blog or follow him on Twitter.

As you could have guessed from the title, this post is about my wishful thinking of software testing. Of course, I can’t be expected to cover all of my wishes in a single blog post, but I have done my best to highlight a few areas of great potential – and here they are:

More active software testing communities
As many of you know from being a member of uTest, software communities will play a big part in the future of testing. Although uTest is a global community, I envision similar communities popping up in every city, state and country. The Weekend Testing group, of which I’m also a member, is another great example of this.

Software testing in schools and colleges

Pradeep Soundararajan – my testing coach – said that kids sometimes ask better questions about a product than what many so-called experienced testers ask. He talked to me about how questions from school kids would lead them to solutions in their own software and products. They asked some very intelligent questions and I was amazed by the perspective they showed. It made me wonder, “Why are students  not taught about thinking (example: lateral thinking) as a skill in most schools and colleges?”

Read more…

To say that uTester Ajay Balamurugadas has an impressive software testing resume would be an obvious understatement. Coached by Pradeep Soundararajan, he has been awarded a scholarship from the Software Testing Club; is a proud student of  the Miagi-Do School run by Matt Heusser, and co-founded “Weekend Testing.” Oh yeah, and he’s also the latest contributor to our guest blogger series. For more of his work, be sure to check out his blog or follow him on Twitter.

In this post, Ajay takes a stroll down memory lane…

This is an article on my experiences with software testing, the traps I fell into, and the lessons I learned in the process. Before I share my story, let me make one thing clear: I’m no software testing expert. I make mistakes, learn, practice and apply my learning to improve my skills as a tester. To illustrate, I’ve split the journey into five simple stages.

Stage 1: Testing = Find Bugs

I am hired as an Associate QA Engineer at my first job. I was called upon to help remove all bugs in the product before it reached the customer – simple enough. As an obedient student, I did what was expected of me. The execution percentage never reached 100%. I could not complete a cycle of execution in the stipulated time. I did not know that I was checking and not testing. Whenever I tested, I could not achieve 100% execution. Some of the bugs I logged were termed as ‘Deferred – Will not be fixed’. I was bombarded with questions like: “Which user would do that? Good bug, but why did you find it now? Why did you miss it? ”

I did not have an answer for the questions. I myself had more questions than answers.

Read more…

How should testers respond when their bugs are rejected by clients?

Here to address this topic is Bill Ricardi of Northern Ireland, this month’s guest blogger. An active member of the uTest community (especially in the uTest forums), Bill describes himself as a “huge nature fiend, with professional ties to advertising, real estate, gambling, and writing.” For more on Bill – including the upcoming publication of his first book – check out his Google Knol page.

In the field of contract software testing, you won’t always see eye to eye with the client. What you consider a critical bug, they might see as a non-issue (or worse, a ‘feature’). What you call a major security flaw, they might consider such a remote possibility that it doesn’t even deserve a mention.

You might ask how you bridge such a gap between your level of testing and the client’s level of acceptance and understanding of product integrity and the testing process in general. The answer is simple:

You don’t.

It isn’t your job to convert the client to your way of thinking. Yes, you can contest a bug that they reject out of hand if you were technically correct to report it. Sometimes they’ll accept it as valuable feedback, but most of the time they’ll just ignore any contested bugs. This is something that you have to live with.

Read more…

So far, our Guest Blogger series has demonstrated the incredible domain expertise of our community – and this post will be no exception. With us this month is uTester David Vydra. A resident of San Mateo, California, David is an Agile Tester for Guidewire Software (he’ll explain what they do in a bit). If his name sounds familiar, that’s because he is the co-author of testdriven.com – a very popular testing site covering developer testing, automated exploratory testing, model-based testing and more.

In this post, David offers us his “notes from the field”, where he’ll be addressing the role of  software testers; criteria for hiring testers; the experts he follows and more. We’re confident that you’ll like it. Enjoy!

I am thrilled to be invited to share my testing experiences with the uTest community. I hope my account will encourage more organizations to adopt the agile way and more testers to find fun and fulfilling jobs.

I joined Guidewire Software about seven months ago as an Agile Tester. We make core software for the Property and Casualty insurance carriers. If you have car or homeowners insurance, you may be serviced using our software. Right from the start, Guidewire has used agile development methods and credits a good part of its success to this philosophy.

Our applications are complex. It typically takes several months for a tester to become fully productive because there is so much domain knowledge and in-house tooling to learn. Our applications are enviably flexible, and each installation is customized to fit the specific needs of the insurance company. In order to empower the customization process, we provide a number of development tools including a custom java-compatible scripting language, an IDE, a GUI framework, and a screen painter.

Read more…

We’re excited to kick-off the new year with Yvette Francino as the latest contributor to our Guest Blogger series. Before joining the uTest community, Yvette  spent the bulk of her career as a developer for companies like IBM, and most recently, as an IT manager at Sun Microsystems. An aspiring novelist, she also publishes “New World Software Quality Assurance”, a blog covering all sorts of testing-related subjects.

In this post, Yvette offers advice for testers who hope to engage in ‘civilized’ dialogue with their developer counterparts. A touchy subject, to be sure!

Software testing is one of the few professions where an unfortunate task is to tell someone else what is wrong with their product!  This, of course, can make the recipient of such news – the developers – a bit defensive. Imagine hearing that your baby is defective! It’s therefore important to employ a bit of diplomacy when delivering such news.  Being a tester is comparable to being a doctor. Let’s listen in on couple of hypothetical conversations. Young Susie (i.e. the software) has been examined to see if she’s ready to enter Kindergarten (i.e. the market).

Conversation with Dr. Tactless

“I have found quite a few problems with Susie,” Dr. Tactless reports to Susie’s parents, looking quite satisfied with his report. “I found 20 problems on her hands and feet. Each toe on each foot has a rash on it. And each of her five fingers on her two hands also has a rash.  What have you been feeding this child? When a child has 20 problems, we can’t let her go to school.  Not only that, but her head is weak. She certainly will die if you let her go to school now.”

“Die? NOOO!”

“When I pushed her down, she knocked her head on the floor, and then she wasn’t able to think. Had I pushed her any harder, she certainly would have died.”

“Why did you push her down?” the parents ask in horror.

“I had to simulate what the kids at school will do.  She is really ugly – uglier than any kid I’ve ever seen. She’s going to be pushed down a lot. She’s going to need major surgery to put a protective layer on her head before I can sign off that she’s ready to go to school. ”

Read more…

As a way to extract the collective wisdom of the uTest community, we decided to experiment with a Guest Blogger program beginning in April. To say that it’s been a success would be an understatement, but we’ll say it anyway (the number of page views don’t lie!). Having covered a wide range of topics – including mobile app testing, tester overconfidence, security testing and more – the series has become a big hit within the community — and a great way for testers to get published in front of a large audience.

Here are a some of the highlights from our 2009 guest blogger program.  Stay tuned for an even bigger 2010!

Who is the User? – by Lucia Maldonado:  In what ways is software similar to architecture? And how can this help steer testers in the right direction? In this post, Lucia Maldonado takes an in-depth look at user accessibility standards, and offers a number of essential tips for testers in this field.

Security Testing Tips (from a Bug Battle Winner) – by Bernard Shai Lelchuck:  When it comes to security testing, few can match the expertise of Bernard Shai Lelchuck – one of uTest’s first (and finest) QA professionals. In this post, Bernard covers the basics methods of security testing, including tips for  information gathering, logical attacks and injection attacks. Oh, and here’s Part II.

Respect the Defect: Advice That Will Change the Perception of  Testing – by Joseph Ours:  Testers need to reconsider they way they report bugs – this was the position taken by Joseph Ours in his first (and hopefully not last) uTest blog post. Challenging testers to demonstrate their value by writing more clearly about the bugs they uncover (among other tactics), Joesph has sparked an interesting debate among our community. Visit the comments section to see for yourself.

Step Away from the Simulator: Putting Mobile Applications Into a Tester’s Hands – by Brad Sellick:  What makes mobile testing different from conventional software testing? For one, the simulators and emulators are far less reliable. In this post, uTester Brad Sellick – a self-made expert on mobile app testing and development – explains the dangers of relying on these tools while performing mobile app testing.

What You Need to Know About Writing Effective Test Cases – by Valerie Dale:  Despite all evidence to the contrary, test case design is often seen as work with no real value – a remedial task with no significant ROI. One would think that with the added pressures to launch a quality product on schedule, test case design and planning would be a top priority. It’s not. At best, there is minimal attention paid to the practice. At worst, it’s non-existent. In this post, Valerie Dale makes a great defense of  this beleaguered practice.

Your Overconfidence is Your Weakness: Lessons from a “Crash Specialist” – by Pradeep Soundararajan:  In our most-popular guest post to date, noted blogger Pradeep Soundararajan explains why finding lots and lots of bugs isn’t necessarily a good thing. Reliving his days as a “crash specialist” Pradeep examines how a tester’s ego can get in the way of their objective. His advice is as funny as it is useful. Simply put: a must read.

Software Testers: The “Eyes of the Battlefield” – by Brian Rock:  Our testers come from all sorts of backgrounds, including the armed forces. Brian Rock – a former Sgt. for Combat Arms Forward Recon Team in the U.S Army – is a great example. In this post, Brian makes analogizes testers with cavalry scouts. That is, they are the “eyes of the battlefield.”  Advocating exploratory software testing (especially for those in the uTest community) this post will make you rethink the role of testers.

You’re a Professional Mobile Tester (you just don’t know it yet) – by Bernard Shai Lelchuck:  As the title would imply, this post makes the case that anyone with a mobile phone and an inquisitive mind can become a successful mobile tester. It worked for Bernard Shai Lelchuck! Here Beranrd explains the rise in mobile applications, how he himself broke into the field and some basic tips for those who would like to get started in this growing (and highly lucrative) field.

Question the Connection: Tips for Diagnosing User Login Failures – by Sherry Chukpa:  Forget the sweeping generalizations about software testing “best practices.” This post by uTester Sherry Chupka gets right to the point on a very specific issue: user login failures. If you’ve ever been pitted against this problem in the testing lab, Sherry feels your pains, and has some invaluable advice for you as you move forward.

It’s been a great year, with some terrific insights into the world of testing, but our Guest Blogger program is just getting started. So if you have an opinion to express, a tip to share or a bone to pick, we’re always eager to share the thoughts of our tester community. Email us your ideas at marketing@utest.com.

User login failures – why do they occur and how can testers spot them beforehand? That’s the latest subject of our Guest Blogger series. Addressing the topic is veteran uTester Sherry Chukpa of Pittsburgh, Pennsylvania. A married mother of three teenage sons, Sherry began her software testing career back in 1998 after obtaining a degree in Applied Science. Aside from being a full-time tester, and an active member of the uTest community, Sherry is also a prolific testing blogger. You can read her blog or you can follow her on Twitter. Either way, testers should be listening carefully to her advice. Enjoy!

This article lists some of the scenarios that cause user login failures. It is my hope that this will give you some new ideas when building future test cases.

What happens if the ODBC Data Source is configured using the incorrect driver version? For instance, I tested an application that was an add-on module to a web application. The add-on module required the Native SQL Server driver. The web application had its own ODBC Driver. The add-on module did not handle names containing spaces if it connected using the web application’s ODBC Driver instead of the Native SQL Server Driver.

What if there is more than one ODBC Data Sources configured to use the same name (one under System DSN and one under File DSN)? I once saw an application that displayed the ODBC Data Source names in a drop down list. The program would only show one of the Data Sources if there was more than one DSN using the same name. You could not be sure which database the program was connecting to.

Does the software handles DSN names containing special characters? The data source name may accept special characters that the application does not handle correctly. Some languages use the ampersand (&) symbol to underline the accelerator key or keyboard shortcut in a menu. The ampersand symbol might be displayed as an underline.

Read more…

When our Guest Blogger series began a few months back, you might recall that it was Bernard Lelchuck who got things started. For those who are new to uTest, Bernard has been one of our top testers from the get-go, and you can read more about his background and uTest experience by checking out his Tester Spotlight. In his latest post, he explains how he got into the lucrative field of mobile app testing – and how all testers can (and should) do the same. Enjoy!

If you haven’t noticed, the use of mobile applications has skyrocketed over the past few years. And while most mobile companies are lagging behind Apple’s success, the market itself has nevertheless become a multi-billion dollar endeavor. As one might expect, this success has prompted competitors of all sorts to rush and open their own mobile application stores. They naturally seek greater market share, and who could blame them?

According to a recent report published on the Wireless Expertise website, “the global mobile app market – including games – will be worth $4.66 billion in 2009, rising to $16.60 billion, in 2013.”

This of course would help explain the sudden entrance of Microsoft, Google, Research in Motion (RIM) and Palm, along with mobile vendors like Verizon and AT&T into the mobile market. As I like to say, they are trying to catch the fast-riding “Mobile App Train.”

And what a ride it’s been! Since the 1^st gen iPhone was released in June of 2007, almost every leading mobile vendor has changed their products to look, feel and be as cool as the iPhone (with varying degrees of success).

Which brings me to mobile testing. But before I discuss the testing implications of this iPhone mimicking trend, I’d like to address how I got into mobile testing in the first place.  It’s my hope that this story will encourage other testers to consider furthering their careers by hopping on board the Mobile App Train.

Read more…