Another month, another stellar guest for our Testing the Limits series. This time, we shoot some questions back-and-forth with testing expert Jim Sivak. Jim has been in the computer technology field for over 35 years, including a recent four-year stint as the Senior QA Manager at McAfee.  His career as a tester began with the Space Shuttle and over the years has encompassed warehouse systems, cyclotrons, radars, operating systems and now security software.  He is a Senior member of the ASQ and is certified as a Software Quality Engineer (CSQE).

In part one of our interview, we get his thoughts on the dangers of ignoring security testing; the false sense of security in mobile apps and devices; the evolution of malware; managing QA expectations; the meaning of SWAG and much more. Be sure to check back tomorrow for Part II.

**************

uTest: We noticed that you recently joined Unidesk after four years at McAfee. First off, what does Unidesk specialize in? And what are you looking forward to most in this new role?

JS: Unidesk is in the Virtual Desktop space. Our product allows companies to utilize virtual desktops that truly have the look, feel and capability of a hardware desktop. Due to our technology, desktop personalizations are easily managed. Virtual Desktops can become the IT department’s best friend in that changes and patches only have to be rolled out to one system, which then gets replicated automatically to every associated desktop.

Because Unidesk is a startup, I have the opportunity to really define the QA processes and goals, determining both the tactical and strategic visions. Being able to drive this work, using new techniques and past experience is really what brings me to my desk every day.

uTest: Your time at McAfee must have given you great insight into the web’s dark underbelly (i.e. security threats). Looking back over the last few years, what’s surprised you the most about the way businesses and consumers deal with security measures?

JS: Great question, Mike. The biggest surprise is the whole ostrich “head in the sand” attitude that exists. The tools and techniques are there, the information is readily available, but security still takes a lower priority until an incident happens.  Just look at the breaches that appear on an almost daily basis.  In the home, how many emails do people open and respond to that say ‘you have a credit card application ready for you”?

uTest: It seems safe to assume that users are more aware of threats on the web, as opposed to mobile? In your view, how does the explosion of mobile apps, social media and third-party integrations affect security?

JS: It is the sheer volume of opportunity for security lapses and breaches with these new avenues that is really frightening.  Just look at the incidents that have happened because someone sent a malicious link to their networked friends unbeknownst to them. Or applications that contain malware that just get downloaded and incorporated on these devices. People just assume that their phone is secure or that their tablet is unhackable. Again, software providers need to take security seriously and not wait until a major incident happens. It all comes down to the fact the users are human and we take a lot for granted.

uTest: Which evolves faster: security threats (viruses, malware, etc.) or the technology used to combat them? Why is this the case and what are the implications for end-users going forward?

JS: Unfortunately, I think that malware is winning.  Although there is research in trying to get ahead of the bad guys, most technology is reactive—the threat exists first and a solution/detection comes after.

uTest: This might seem like a job interview question, but what were some of the biggest testing challenges you faced at McAfee and how were you able to overcome them?

Read more…

Luminary: a person who has attained eminence in his or her field or is an inspiration to others

You’d be hard-pressed to find a profession with a wider range of ideas and personalities than that of software testing. This point is certainly not lost on our readers, as evidenced by the popularity of our Testing the Limits interview series. And it’s not lost on our good friends at Software Test Professionals, who have opened up nominations for the 2nd Annual Software Test Luminary Award.

More on the nomination process in a second, but first, a little bit about the award itself:

The Luminary award will honor any software testing and quality assurance professional who is determined, persistent, and committed to improving a process or methodology. They develop ideas, which when properly applied, have a positive impact on the end product, either by enhancing quality or performance or resulting in improved efficiencies for a particular process, team or organization. In addition, their contributions elevate the critical role of the software test profession within the software development process.

A luminary is someone who has inspired others by their actions and the results of those actions on the profession. They inspire others to pursue a software testing career. It is about how they have given back, and shared their knowledge and experience with others in order to advance the profession and improve the career paths of all practitioners. A luminary will typically be recognized and respected long after their days of practicing have ended.

If you recall, last year’s honor went to Gerald M, Weinberg, who edged out fellow nominees James Bach and Cem Kaner.

So who will be named this year’s Software Test Luminary? It’s your call. STP will gather nominations and submit the top 3 candidates for a final round of voting. The finalist will be announced at the Software Test Professionals Fall 2011 Conference, October 24-27 in Dallas, Texas.

Here’s a quick timeline of the events:

Read more…

In part II of our Testing the Limits interview with Jon Bach, we get his thoughts on responding to change in the testing world; what his brother James Bach has been up to; his criteria for hiring testers at eBay; mobile challenges; searching for defective pocket change and more. If you missed the first session, you can read it here.

uTest: It looks like eBay wasn’t able to keep you off the testing speaker circuit (woohoo!). In fact, you were at STPcon earlier this month – care to give our readers who couldn’t make it a summary of what you covered?

JB: Two things: A workshop with Dan Downing of Mentora, who approached me at the last WOPR (held at eBay in November) and had a cool idea to bring a little slice of WOPR to STP.  It’s for anyone who needs to build a game plan for performance testing.  He called it “Arming Yourself for Performance Testing: War Stories from the Trenches” — http://www.stpcon.com/Item/1032/.

I also spoke about an idea that I’ve been experimenting with after James came back from a business trip and talked about how to respond to project change and chaos: http://www.stpcon.com/Session/13/My-Crazy-Plan-for-Responding-to-Change

uTest: Speaking of James, he has been doing some interesting things the past year as well. What’s the latest testing topic of conversation among the Bach brothers? And did he have any words of advice for you in starting your new job?

JB: He came to eBay and spent a week with me.  I sat him in the cube next to mine and he did some testing from outside the firewall on the guest wireless. I gave him a charter and he executed it beautifully.  The secret about James is, he’s really friendly and service-minded if you’ve managed to win his respect.

We talked CAST 2011 (I’m conference president, he’s my program chair); we talked about new tester games; we shot a new CAST promo video; we talked about Egyptian democracy and systems thinking (how it affects the price of gas).  But just when we were in the thick of testing eBay site page compatibility with IE 9.0, the Japan quake hit and we took time to watch the footage with the rest of the world.  Then we did impromptu research and found out more on nuclear plant  meltdowns, which led to being curious about microseiverts, which led to an article about Byzantine failures.

About eBay, he gave me no advice per se, just ideas for tactics.  He offered some free consulting, which he gave, then said, “I’m proud of you, man. Kick ass.”

uTest: Part of your new role at eBay will be to hire and recruit a top-flight team of testers (in addition to the ones already there). What sort of traits/skills/attributes will you be looking for in particular?

JB: The ability to come up with ideas – either old or new – and execute them in a way that helps us improve notions of Search.  For years, I used the triangle program in test auditions.  Now I use something more simple.  I draw a long horizontal rectangle on the whiteboard with a little “Submit” button below that.  I say “this is a text input field for Search, just like the one you see on the eBay site. Help me create a test plan for it.”  I’m hoping that instead of an interview, it comes across more like an invitation to a real collaboration.

Read more…

A great time had by all at STPcon 2011 down in Nashville, Tennessee. For those not able to attend (and those who were not following STP’s live stream) I wanted to post a brief summary of the event. Hey, it’s the least I could do.

Though I tried, I was not able to attend every session, so this is far from a complete report. That said, here were a few major themes I noticed throughout the week:

Mobile Is Mainstream

The subject of mobile app testing is getting more and more recognition in the testing world. A sign of the times, to be sure. I started the week by attending a presentation by Rama Krishna Pagadala, the Software Design Engineer in Test for Microsoft, who spoke about the need for real-world testing of mobile applications. He covered many of the same topics addressed on this blog (and mobileapptesting.com), including the unique factors that make mobile a much greater testing challenge. Specifically, variables such as low battery, connection speed, continuously running apps, OS, screen size, user preferences and others.

Rama talked a bit about the performance testing he did for the Microsoft Office Communicator Mobile application for Windows Mobile 6.x phones, including one set of automated tests that measured six performance metrics for 20 different scenarios. Fascinating stuff.

Then there was Karen N. Johnson, consultant and author of Beautiful Testing, who’s second session dealt with Functional Mobile Testing. Here, Johnson used a great series of screen-shots to show how simple web tasks like sign-in, log-out and others are exceedingly complicated when it comes to mobile. She was also very careful to stress the importance of manual testing over automated options, saying something to the effect of, “if you don’t know what you want tested, how are you going to automate it?” Good question, right?

Read more…

A spring tour that would make The Rolling Stones proud, here’s where you can find us either presenting, keynoting or exhibiting over the next several months:

• CTIA – March 22/24 in Orlando, FL (Booth 3194/Apps World)
• STPCon – March 22/24 in Nashville, TN (Keynote!)
• SXSW – March 11/15 in Austin, TX
• Star Testing – March 16 in London
• QUEST – April 4/8 in Boston, MA
• Techonomy3 – April 5 in Tel Aviv

If you want to meet up with someone from uTest at any of these events, be sure to email us at marketing@utest.com. Hope to see you there! More in-depth information on each conference, where we’ll be, and on the uTest presentations after the bump.

Read more…

The good folks over at Software Test Professionals want to remind you about a very important election this Fall. No, we’re not talking about the U.S. Congress. And no, we’re not referring to American Idol either (at least not in this post).  Instead, we’re talking about something lasting and meaningful: the 1st Annual Luminary Award.

As described on their award page, this honor will “recognize a person in the software testing and quality community, who inspires others and dedicates their career to industry advancement.” The organizers were looking for someone who has dedicated their career to the betterment of software testing and quality; who has shown exceptional leadership and who has educated, promoted and published on behalf of the industry. In other words, a software testing luminary.

With that type of criteria in mind, we’re not surprised to see Cem Kaner, James Bach and Jerry Weinberg as this year’s finalists. You may know Kaner and Bach from our recent Testing the Limits interviews (Jerry, if you’re reading this, we’d love to have you as a guest as well). But in case you’re unfamiliar with these testing giants, here are clips from their award bios:

Read more…

Quality (pun intended ) software testing events are hard to find, but we’ve not only attended and spoken at some fantastic conferences around the world, but we’ve also simply asked around and received some great feedback in order to compile the Top Ten Testing Events.

Much like our Top 20 Software Testing Tweeps post, we need your help in letting us know if we’ve accidentally missed any good ones. Here they are in order of occurrence:

1. QUEST-Quality Engineered Software & Testing Conference (Apr 19-23, 2010: Dallas, TX)
2. Rapid Software Testing-By DevelopSense (Jul 5-7, 2010: Amsterdam, NL)
3. STANZ-Software Testing Australia/New Zealand (Aug 23-24: NZ & Aug 26-27: AU)
   
4. CAST-Conference of the Association for Software Testing (Aug 2-4, 2010: Grand Rapids, MI)
5. STAREAST (passed) & STARWEST-Software Testing Analysis & Review (Sept 26-Oct 1, 2010: San Diego, CA)
6. iqnite events-Next one in UK-formerly Software & Systems Quality (Oct 4, 2010: London, UK)
7. STPCon-Software Test Professionals Conference (Oct 19-21, 2010: Las Vegas, NV)
8. GTAC-Google Test Automation Conference (Oct 28-29, 2010: Hyderabad, India)
9. Expo: QA (Nov 16-18, 2010: Madrid, Spain)
10. EuroSTAR (Nov 29-Dec 2, 2010: Copenhagen, Denmark)

Have we omitted any noteworthy testing conferences you’ve recently attended? Please add your recommendations in the comments and they’ll be placed in the running to join the top events list. Maybe we can make this list a Top 15!

UPDATE: So far, some really great recommendations from our community include O’Reilly Velocity, Bangalore Workshop on Software Testing and VISTACON 2010 (the first Vietnam International Software Testing & Automation Conference).

Retired NASA Astronaut Mike Mullane* (pictured left) said it best when he asked: “Why is there never time to do it right, but always time to do it over?” He could have easily been talking about the recent problems Toyota has been dealing with, but he wasn’t. He was talking about today’s software companies.

Conversely, this recent article from The Economist could just as well be about today’s software companies, but it isn’t. It is about Toyota’s recent problems.

Like everyone else, the author wants to know how the auto giant could so quickly lose its reputation for safety and quality (things that can happen to ANY company if they are not careful). The culprit? You guessed it: software bugs.

Instead (of trying to keep pace with competitors), two recent trends, both software related, hint at the reason behind Toyota’s unexpected decline. One is the shortening of product-development cycles generally in the car industry. These are down from a typical four or five years to little more than 15 months, thanks to computer-aided design and manufacturing, and the virtual simulation of the resulting products. To save money and time, Toyota has even dispensed on occasion with building test “mules” and other engineering prototypes.

Read more…

In the first part of our interview with Michael Bolton, we grilled him on the emergence of the Weekend Testers, sensible metrics, Michael Bolton the pop star and a bunch of other topics. In part “deux” of our interview, we tackle the necessity of tester passion, how emotions affect testing, and the greatest threats to the profession. Check back tomorrow for the final segment.

uTest: There’s a lot of passion amongst testing thought leaders about the best way to test, or the best way to manage or train testers.  Often that passion overflows into heated debates.  How can this passion best be channeled to improve the state of testing?

MB: First of all, we should welcome debate, and we should welcome skilled argumentation as part of the art of construction and practice of persuasion. I’ve found, though, that it helps to remember that we’re exploring and challenging ideas. That means it’s good not to get too personally invested in certain ideas, because we’re always learning more, and because changes in context can mean big changes in what needs to be done.

That said, there are some ideas that seem robust for me. I believe that it’s unethical to dumb down people or the work that they do. I believe that we should focus our craft on learning, and learning how to learn rapidly. How can we improve the state of testing? By recognizing that software development is a web of people who are related in service to each other. That means putting people and social issues first. Get that right, and everything else will follow.

Suggestions are cool.  Standards are something else.  No group should be dictating to other people how they must test unless there are compelling human health and safety reasons for it. Do you really believe that the standards people know anything useful about your business? That the force of government-supported regulation, created by busybodies, should weigh on how you do your daily work? And if your answer is No, what are you going to do to get it stopped?

Read more…

In this month’s installment of “Testing The Limits”, we sit down with Matt Heusser (@mheusser) — prolific blogger for STPCollaborative, thought leader and testing extraordinaire.  We’ll discuss the state of software testing, SpeedGeeking, the role of chaos in testing software, and the lack of fistfights at STPCon 2009

uTest:  We loved the SpeedGeeking session you led at STPCon, so we’re going to flip it on you – If you had just five minutes to teach, motivate or inspire the uTest audience about software testing, what would you say?
MH: Well, I’d start by asking the audience what they are doing today – what’s the greatest point or opportunity they feel – and asking what options they see to improve. Most of the time, I hear that testing is “too slow” or “the bottleneck” or something like that.

So I suggest taking two weeks and actually measuring how the team is spending its time. Oh, not for reporting – it is very important the team stop the time tracking after two weeks and not hand individual metrics into management for evaluation. Instead, we want to use the numbers for improvement. For example, many of the people I talk to can spend 80% of their time or more in meetings, working on documentation, working on compliance activities, doing email, and so on. That only leaves 20% of the time to test! Just pushing those numbers from 80/20 to 60/40 will double the amount of time the team spends actually doing testing.

Another thing to look at is the amount of time spent trying to reproduce defects, document defects, file bug reports, “verify” fixes, and so on. We think of these activities as testing, and they can take a substantial chunk of that 20% – but they are really accidental. That’s not a testing bottleneck – it is a development bottleneck. If test can work with development to improve the quality of the software prior to code complete, that will improve the speed of the whole system. Realizing this, and having a little bit of data to “prove” it, can help the entire system improve.

So if I had five minutes, I would say start with measuring how you track your time, and ask yourself if this is the best use of your time and what can change. Sometimes, the big boss will say “no, we absolutely need you to fill out all seven pages of documentation per test run”, and you can say “ok.”  Six months from now, when someone asks why the big project is late, you can point out that the business made an explicit decision to pay the full price of defined process. You presented options and those were not accepted.

That won’t save this project — but it might save the next.  It also turns out that actually testing tends to be much more fulfilling than documentation and compliance activities. Who could have guessed?

Lots of contrasting opinions at last month’s STP Conference. While there were no fist fights (that we heard about anyway), what did you see as the most contentious issue? And where do you fall on this issue?

Read more…