So what’s the proper mindset for a software tester? Much has been written on the topic – and it’s critical to your success in the field – so I figured I’d offer a view different points of view.

Here’s one from softwareprojectmanager.com:

A professional tester approaches a product with the attitude that the product is already broken – it has defects and it is their job to discover them. They assume the product or system is inherently flawed and it is their job to ‘illuminate’ the flaws.

This approach is necessary in testing.

Designers and developers approach software with an optimism based on the assumption that the changes they make are the correct solution to a particular problem. But they are just that – assumptions.

Without being proved they are no more correct than guesses. Developers often overlook fundamental ambiguities in requirements in order to complete the project; or they fail to recognise them when they see them. Those ambiguities are then built into the code and represent a defect when compared to the end-user’s needs.

Here’s another view from onestopsoftwaretesting.com:

Pedantic, sceptical, nit-picking to software

Some years ago, we were asked to put a slide together, saying who makes the best testers, and we thought and thought, but eventually, all we could think of was, they’ve got to be pedantic and sceptical and a nitpicker. Now, if you called someone a pedant, a sceptic, and a nitpicker, they’d probably take an instant dislike to you. Most folk would regard such a description as abusive because these are personal attributes that we don’t particularly like in other people, do we? These are the attributes that we should wear, as a tester, when testing the product. When discussing failures with developers however, we must be much more diplomatic. We must trust the developers, but we doubt the product.

Most developers are great people and do their best, and we have to get on with them – we’re part of the same team, but when it comes to the product, we distrust and doubt it. But we don’t say this to their faces. We doubt the quality of everything until we’ve tested it. Nothing works, whatever “works” means, until we’ve tested it.

Impartial, advisory, constructive to developers:

But we are impartial, advisory and constructive to developers. We are not against them, we are on the same team. We have to work with them, not against them. Because it is human nature to take a pride in their work and take criticism of their work personally, bear in mind this quote: ‘tread lightly, because you tread on their dreams’.

Here’s a similar view from kualitatem.com:

All software must possess bugs as the developers are human and you can’t eliminate the error factor from humans. Therefore it’s very important for the tester to think from end user perspective and try to find out all the possible bugs that a user can face. The approach to use Mindset Testing is to treat the software as it’s full of bugs; the developers deployed an application that will crash when end-user will use it and testing its all possible flows. Here begins the work of a tester to find out all these bugs and loopholes that will make this application go crashing.

Don’t take any bug lighter, “Report all bugs you have encounter with”, as you will be responsible for it at the end.  Although sometimes with the mutual understanding of the developers and QA, some bugs are not addressed right away, but still, from QA perspective one should not skip reporting any bug.

What do you think? Is there a proper mindset for testing? If so, what is it? Please share your thoughts in the comment section below.

The Sport Evac program trains teams for those “what if” scenarios, by creating virtual 3D stadiums drawn from actual blueprints and packing them with up to 70,000 animated human avatars designed to respond to threats as unpredictably as their human counterparts.

It’s advanced technology fans won’t see on game day — but tech that behind the scenes makes watching the big game safer.

Evacuation in the event of an emergency is a critical challenge, and rehearsing the how to move around tens of thousands of people isn’t realistic. So sports security planners turn to computers to make sure it all goes smoothly.  …

The virtual stadium allows them to simulate how fans will respond in those first few critical minutes after an attack. Stadium and team security can use the virtual stadium to practice moving players and fans to safety and to run exercises with local first responders. …

As with any test, the ability to scale to meet different demands is key to success. Only testing for the “best case scenario” isn’t very effective.

Sport Evac isn’t the only evacuation software program, but others have struggled to meet the scaling challenge of sporting events. Other simulations also failed to include the wide range of human behavior variations and the multitude of factors that may disrupt the best-laid aid plans.

All sorts of curve balls can appear in a threat: parking gridlock, for example, making an orderly exit disorderly, or the elderly having a hard time with a crushing crowd.

Sport Evac allows security teams to test the robustness of their planning against the full range of possible factors — fans fighting against the stream to retrieve forgotten wallets and handbags, spilled beer creating too-slick floors, emergency lights failing and so on.

Not only did the makers of Sport Evac create and test desktop software, they are also working on a mobile app version of the software for stadium employees and first responders to use on-site. This app sounds like a perfect candidate for some in-the-wild testing because any sports fan can tell you that the middle of a sold-out stadium isn’t the best place to get a signal. Plus, this is the last app you want to crash, freeze, load slow or glitch at the exact moment you need it most.

A smartphone app, known as Sport Evac Lite, will become available as well, so security staff and ushers can see where fans and cars could bottleneck.

Read the full article at FoxNews >>>

With that in mind, I wanted to draw your attention to Classic Testing Mistakes by Brian Marick. Whether you’re a tester or manager, experienced veteran or wide-eyed newbie, this 25-page article outlines some of the most classic testing mishaps and offers valuable tips on how to avoid them.

So what are some classic testing mistakes? One would be not reading this document in its entirety. As for the others, here are a few clips I found interesting. Note the bold titles are mine, everything in italics in Brian’s.

Mistake: Testers Not Responsible for Usability
If usability problems are not considered valid bugs, your project defines the testing task too narrowly. Testers are restricted to checking whether the product does what was intended, not whether what was intended is useful. Customers do not care about the distinction, and testers shouldn’t either.

Mistake: Misunderstanding the Role of “QA”
A first major mistake people make is thinking that the testing team is responsible for assuring quality. This role, often assigned to the first testing team in an organization, makes it the last defense, the barrier between the development team (accused of producing bad quality) and the customer (who must be protected from them). It’s characterized by a testing team (often called the “Quality Assurance Group”) that has formal authority to prevent shipment of the product. That in itself is a disheartening task: the testing team can’t improve quality, only enforce a minimal level. Worse, that authority is usually more apparent than real.

Mistake: Bad Timing on Load Testing
Putting stress and load testing off to the last minute is common, but it leaves you little time to do anything substantive when you discover your product doesn’t scale up to more than 12 users.

Mistake: Relying on Beta Testing
Beware of an overreliance on beta testing. Beta testing seems to give you test cases representative of customer use – because the test cases are customer use. Also, bugs reported by customers are by definition those important to customers. However, there are several problems:

1. The customers probably aren’t that representative. In the common high-tech marketing model4, beta users, especially those of the “put it on your web site and they will download” sort, are the early adopters, those who like to tinker with new technologies. They are not the pragmatists, those who want to wait until the technology is proven and safe to adopt.

Mistake: Having Programmers Test
Using testing as a transitional job for new programmers is one of the two classic mistaken ways to staff a testing organization. It has some virtues. One is that you really can keep bad hires away from the code. A bozo in testing is often less dangerous than a bozo in development. Another is that the developer may learn something about testing that will be useful later. (In my case, it founded a career.) And it’s a way for the new hire to learn the product while still doing some useful work.

Mistake: Testers Not Domain Experts
Be especially careful to avoid the trap of testers who are not domain experts. Too often, the tester of an accounting package knows little about accounting. Consequently, she finds bugs that are unimportant to accountants and misses ones that are. Further, she writes bug reports that make serious bugs seem irrelevant. A programmer may not see past the unrepresentative test to the underlying important problem.

Mistake: Poor Bug Reporting
It’s not enough to find a failure; you must also report it. Unfortunately, poor bug reporting is a classic mistake.

Mistake: Unrealistic Expectations
Whatever approach you take, don’t fall into the trap of expecting regression tests to find a high proportion of new bugs. Regression tests discover that new or changed code breaks what used to work. While that happens more often than any of us would like, most bugs are in the product’s new or intentionally changed behavior. Those bugs have to be caught by new tests.

What other common testing mistakes have you experienced? Please share in the comment section.

This according to our very own CMO Matt Johnston, who recently sat down with Rich Hand of softwaretestpro.com for an in-depth discussion on the growing importance of in-the-wild testing. This lively Q&A was then adapted into a featured article  (no word yet on its big screen debut).

Anyway, if you’re new to the concept of in-the-wild testing or want to know more about it, then I highly recommend giving it a proper read.  Here are a few clips to get you started.  Enjoy!

Does it ever seem that no matter how much time, effort and money your QA team spends improving and refining software testing processes that some sort of defect is always found in the application, website, or mobile app after launch? Surprisingly, it has little to do with your organization’s in-the-lab testing – whether in-house or outsourced, manual or automated. In fact, it’s likely due to the fact that the lab environment you’re testing in cannot adequately replicate real-world conditions (real users, real devices, diverse locations, imperfect connectivity, not to mention a range of devices, operating systems, browsers, etc.).

And thus, companies test extensively in the lab, launch their apps into the real-world conditions of users, and we’re all surprised when these products don’t perform as expected. But what kind of testing can fill this gap effectively and affordably? It was out of this persistent question that uTest and crowdsourced testing was born. And the by-product of this was a new category of testing which has become a must-have for mobile, social and local apps known as “In-The-Wild Testing.”

“In-The-Wild Testing” (ITWT) is an effort to educate tech leaders about how to help QA teams and organizations launch higher quality software, quicker, faster, and cheaper. The idea of in-the-wild testing is about providing organizations with the real-world testing data necessary to make informed decisions about releasing products to market. According to Matt Johnston, Chief Marketing Officer for uTest, “Don’t be fooled by the word ‘wild’ when it comes to testing software. When you think of the term ‘In-the-wild testing’ think of it as ‘real-world vs. laboratory conditions.’” This is not outsourcing or beta testing, and it’s definitely not suggesting you replace the QA teams or solid processes you have in place within your test lab. Rather, this is about complementing, scaling, and aligning professional testing resources with your in-house or outsourced QA team. I predict that this concept will explode in the next five years . But the first step is to understand what ITWT is (and isn’t).

Continue reading >>>

The Sidney Morning Herald explains how this relates to software testing:

Unlike the better-known leap year, which adds a day to February in a familiar four-year cycle, the leap second is tacked on once every few years to synchronise atomic clocks – the world’s scientific timekeepers – with Earth’s rotational cycle, which, sadly, does not run quite like clockwork. The next one is scheduled for June 30 (do not bother to adjust your watch).

The United States is the primary proponent for doing away with the leap second, arguing that these sporadic adjustments, if botched or overlooked, could lead to major foul-ups if electronic systems that depend on the precise time – including computer and cellphone networks, air traffic control and financial trading markets – do not agree on the time.

Abolishing the leap second “removes one potential source of catastrophic failure for the world’s computer networks,” said Geoff Chester, a spokesman for the US Naval Observatory, America’s primary timekeeper. “That one second becomes a problem if you don’t take it into account.”

By now, you’re probably wondering what the “debate” is all about. Is anyone voting in favor of catastrophic failure? On the other hand, how can a unit of time be abolished, even if it’s only a second? The story continues:

But Britain, along with Canada and China, would like to keep the current system, arguing that, in the 40 years that leap seconds have been gracefully inserted in our midst – most recently in 2008 – there have been no problems to speak of, and the worriers have greatly exaggerated the potential for havoc. Remember Y2K?

“It’s the devil we know,” said Robert Seaman, a software engineer at America’s National Optical Astronomy Observatory. While he is an American, he is also a member of another group wary of the change: astronomers. If a software-guided telescope is not pointed in the right direction, it may not capture the right image, and updating software could be a sizeable task.

Since the 1950s, the world has run on two sets of clocks. One is the ticking of atomic clocks, defined by the precise frequency that electrons jump around in atoms. The other is based on the traditional notion of a spinning Earth.

Anyway, I wanted to bring this story to your attention for two reasons. One, I find it fascinating. And two, to find out what experiences you’ve had with small bugs that led to big mistakes. So please, share your experiences in the comment section below….and be sure to take your time

“Today, far too many of our graduates are leaving without the skills they need to succeed beyond high school. Not every student wants to go to college, nor is college right for everyone. But all students should leave prepared to succeed in the next phase of their lives,” Bloomberg said. “It’s a new way of thinking about secondary school based on today’s economic realities.” …

Frank Thomas, a spokesman for the city’s Department of Education, anticipates that the school will have between 420 and 460 students by 2015, when all four grade levels are enrolled, Adrianne Jeffries reported in BetaBeat. The school will start with a ninth-grade class this year and add on another grade level for the next three years.

The city has other specialized high schools for science, math, the performing arts and other subjects, but it did not have one focused on computer science. …

Joel Spolsky, a board member of the new school, said one reason he’s a proponent of the school is that it could can train many excellent software engineers who are not currently at the top of their class academically.

“I think this is the best thing about the school,” he said in a blog post. “A lot of kids are just not interested enough in other academic subjects to get good grades, but they would make great software engineers. A lot of immigrants (especially in New York) are not yet proficient enough in English to get good grades in all their subjects, but they’re going to make great software engineers, too.”

I have to say, one instance is cool. Two instances make you raise an eyebrow. Three instances (especially when they’re consistently bigger examples) might just be the start of a trend. And this trend of focusing not only on computer basics, but on more advanced – more engaging – computer topics that can lead to lucrative, fulfilling career paths is long over due.

And speaking of correct answers (or lack thereof) it seems that a software glitch caused 90% of would-be pilots to record failing grades on their online exam. Why online, you ask? In response to the recently exposed “fake pilot” scam – and as an added measure to prevent forgery – the Directorate General of Civil Aviation (DGCA) decided to make candidates take the test online.

Simple enough? Here’s NDTV.com with how this plan went, um, off course:

A software glitch caused a 100-mark examination to be graded on just 50, causing over 90 per cent of the examinees to believe that they had failed.  “The paper had 50 questions worth two marks each. Every candidate had to secure a minimum of 70 per cent marks to qualify.

During evaluation, the online software wrongly assigned one mark to each question, as a result of which each candidate was examined on 50 marks. The merit list, however, showed the full marks as 100. As a result, none of the 1,000 pilots who appeared for the ALT examination managed to pass the exam. Passing the exam is mandatory for officers who are seeking licenses as commanders,” said a top DGCA source.

In the CPL category for commercial piloting licenses, only two-three percent of over 4,000 candidates managed to clear the examination.

Realising that the mass failure was a result of a software problem, DGCA chief E K Bharatbhushan wasted no time in declaring that the incident was a consequence of a software glitch, and said that a new merit list would be published this Monday.

Let’s hope the DGCA  learned their lesson on the importance of in-the-wild software testing. If not, I think it’s time to take the train.

If that wasn’t enough, now you have to make sure that website works on the miniature screens of mobile devices (which themselves offer a whole gamut of sizes). And I’m not taking “should work” or something to consider if you want to be hip and trendy … because it’s not a trend, trends go away. Instead, time spent browsing the web on mobile devices is steadily increasing. Here’s the most recent statistic from Net Applications (which has been monitoring web usage across their 40,000 websites since 2004), as reported by CNet:

If you haven’t whipped your Web site into shape for easy viewing on small-screen devices, you’d better get cracking.

That’s because the use of mobile devices reached an all-time high in December, accounting for 7.7 percent of browser usage according to Net Applications’ measurements of daily visits to its network of 40,000 Web sites. That may still be a small fraction of total Web traffic, but it’s a large and growing population in absolute numbers.

Tablet browsing in many ways is similar to desktop browsing; screen resolution on the dominant iPad and iPad 2 aren’t that far off a laptop. But touch interfaces are different from mouse interfaces, especially when it comes to tapping buttons with precision. And smaller tablets are awkwardly in between the iPad and mobile-phone screens.

Eight percent may not seem like a large amount, but if you look at the chart there’s been some pretty steady and noticeable growth since November 2010. Another CNet article breaks down the popularity of individual mobile browsers and devices. And even in that realm there’s been some shifting and changing – which makes it hard to optimize your website “just for the most popular mobile browsers.”

Despite a drop in December, Apple’s iOS remained the top platform for mobile browsing, according to the latest stats from NetApplications.

Throughout December, iOS carved out a 52 percent share of the mobile market, down from 54 percent in November and 61 percent in October, but still in the lead. Google’s Android ended December with a 16 percent share, a dip from the prior two months.

Drilling down further, the iPhone accounted for 25 percent of all mobile browsing and the iPad 24 percent, leaving the iPod Touch with 2 percent. Android 2.3 was the most popular version of Google’s OS in terms of mobile browsing, followed by 2.2 and 2.1. Devices running Honeycomb 3.2 and 3.1 picked up a 1 percent share. …

Looking at specific devices, Samsung’s Galaxy Tab grabbed almost half a percentage point. And though it’s been hot among consumers, Amazon’s Kindle Fire accounted for just one-tenth of a percentage point in the mobile browsing world last month, according to NetApplications.

But wait, there’s more!

Let’s not ignore feature phones, which still dominate the international market. Java ME is the second most popular mobile browser, having lost the title to Android in October 2011 then regained it. Check out what CNN.com has to say:

Oracle’s (ORCL) Java ME is now the world’s fastest growing mobile OS, having retaining the No. 2 spot it lost briefly to Android in October.

Java ME?

Yes, Java Micro Edition, the Java platform designed by Sun Microsystems and acquired by Oracle in 2010. It’s used primarily in embedded systems, such as the low-cost feature phones (A.K.A. “dumb” phones) sold by the hundreds of millions to people all over the world who can’t afford or don’t want a smartphone.

I know this can be a bit overwhelming. Even if you’re one of the largest companies in the world, who has time to test all these browsers on all the web-capable mobile devices in the market these days? Don’t worry – we do!

If only there were a way that one could test software and burn calories at the same time. Well, now there is. Here’s a funny story from The Register:

No self-respecting techie would ever be seen dead in a gym. If you want to get fit, just put down the doughnut and go for a walk. However for those of you who do submit to treadmill workouts but don’t want to miss a minute’s coding time, this might interest you.

Brainy old Reg friend Bill Softky has come up with an ingenious low-tech contraption for people who want to use a conventional laptop while at the gym. The CardioDesk cracks the problem of providing a stable surface, fits most treadmills and folds away to the size of a book- it’s about an inch thick.

The prototype, which you can see here, is hardwood, but with a bit of interest (and a bit of investment) we should see a much cheaper carbon fibre version to go into production.

He admits it’s “an unusual solution to a nearly unknown problem”, but some of the most successful inventions are just that.

As a side gig, I think I’ll create a promotional series that’s half testing (advice and tips from people like James Bach, Michael Bolton and others) and half exercise (workouts from Richard Simmons and Chuck Norris). I’ll call it TesterCise. It’ll make millions.

On a more serious note, if you’re looking for ways to offset the ill effects of sitting at a desk all day, here are some tips from WebMD.

Test healthy my friends.

So with that in mind, here are 12 questions whose  answers will shape the app universe (and thus, the testing landscape) in 2012:

1. Will we finally find a better way to vet apps than app store ratings?
2. Is Flash really and truly dead in the mobile app space?
3. What’s the next big wave in the ever-growing sea of SoLoMo?
4. Web-enabled TVs:  here or hype?
5. Will Android keep winning such rapid market share from iOS?
6. Is this the year the mobile wallet hits the U.S. mainstream?
7. How will netizens find what they need — search or social?
8. Can developers finally forget about IE6?  How about IE7?
9. Will Amazon’s app store plans fly or flop?
10. Where do tablets go from here?
11. Which direction will the IPO and VC markets turn?
12. After watching Uber battle taxis, and AirBnB take on hotels, which mature industry will be next to get disrupted in a big way (fwiw, my money is on medical and education, though the latter may take longer)?

So what’s your take — which of these issues will have the biggest impact on devs, testers and users in 2012?  Put on your fortune telling hat and share your prediction to that question in the comments below.

And happy 2012 to us all. Let’s enjoy this next (last?) year in the apps universe!