Most of the time, if we’re referencing an interview here on the uTest blog, it’s generally one from our Testing the Limits series. Today will be an exception, thanks to an amazing interview with Bill Curtis on ComputerWeekly.com.

Bill’s central argument – and it’s not an argument you hear very often - is that “the pace of change in the IT industry is hampering software quality.” Not progressing quality, not enhancing quality, but hampering quality. Think about that for a second. When you’re done, take a look at a few of his key excerpts (in italics), followed by my commentary.

BC: “Every five years we have a major change in technology, so every time we get good at building quality code, we change the game. There are new technologies and new languages. You make more mistakes because you are on are on a learning curve. We need a discipline for adopting new technology.”

Couldn’t agree more! Consider the testing landscape from five years ago, 2008. Mobile apps were still very much in their infancy, BlackBerry was still a dominant player, HTML 5 had yet to make much of an impact, Flash was here to stay – I could go on. The point is that times change! Thus, the idea of a discipline dedicated to adopting new technology is indeed quite interesting and would certainly have value (and appeal) for QA teams.

The hard part, I suspect, would be in determining which technologies are worthy of such an exercise, as you would run the risk of teaching an entire department a technology or programming language that never fully takes hold. Also, who makes the call? The QA manager? The product team? The CTO? Obviously there would be a lot of details to iron out, but it’s an interesting idea nevertheless.

Read more…

Despite the United States Postal Service’s reputation as an old-school, clunky organization that doesn’t changed much the Service has been making some giant strides in software initiatives lately. Over the past few years USPS has been embracing new technologies with whole-hearted enthusiasm – and producing noticeable results.

For the longest time, USPS was shackled to waterfall development practices that resulted in slow, costly projects that often failed to meet user needs. The Service’s systems were disjointed and hard to use – resulting in a high chance of error and frustration.

Since piloting Agile projects in 2010, though, USPS has found a better way to complete projects in a timely, cost-effective manner. Even better, their efforts have resulted in improved customer satisfaction. USPS’ mobile app has a higher Applause Score than than Fedex and UPS. This dedication to user satisfaction has been a driving force behind the service’s new approach toward development. As part of the switch to Agile, the Postal Service began incorporating customers into its dev process. These potential customers provide feedback throughout the process and help the organization make sure the service its creating is staying true to customer needs. From FCW:

The [project] incorporated fast deliverables and daily scrum sessions between IT officials and customers to ensure parties were always on the same page. In waterfall approaches, “IT goes off and builds something that 12 months later might not meet the customer’s expectations,” Edgar said, noting that in times of austerity and rapidly changing technology, that approach rarely makes sense.

“The intent was to find a way to deal with changes where we could be more responsive to changing customer expectations and raising the visibility of what we were doing throughout the development’s lifecycle back to our business partners,” Edgar said. “We wanted to better partner with our business organizations, and maintain a more consistent and visible engagement between teams where we could prioritize requirement building so we get the most important things done first.”

Another reason USPS is finding success with its new Agile approach is the roll-out design. Instead of creating a new system as a whole and rolling it out to all customers, USPS has divided its client base for some projects into size-based segments. The project discussed int the FCW article was completed and rolled out to large businesses first to see how companies would respond. Now that the Postal Service is happy it’s beginning to bring the service to smaller organizations. This segmented, rolling approach helped USPS “complete” the project faster and more effectively.

USPS officials said they are very pleased with the results, especially in regards to cost avoidance. …

“What made agile so valuable to us was that it had a hard deadline, whereas if we used a waterfall methodology, we’d probably only just now [June 2013] be planning on doing something,” said Cathy Moon, who formerly managed MTE at USPS. She now works as a manager for USPS’ Operations Integration and Support.

Part of the reason USPS is finding so much success with this new development model, in my opinion, is the fact that they’re focusing on the customers. Sure, these shorter deadlines are helpful and give teams something to aim for and focus on, but a deadline is just a deadline. If the Postal Service hadn’t integrated customer interaction so deeply into the projects they very well could have ended up with fast turnaround projects that were just as disappointing and dis-jointed from user needs as ever. The focus on the customer is the real value-change here.

 Design teams work with customers “almost daily,” and communication between parties is integral to a project’s success. …

“Agile is a way we deliver solutions to our customers, and get those solutions faster and with a more successful conclusion to users, wherever they may be,” Edgar said.

Read the full article at FCW >>>

You know you’re not going to meet your deadline, you haven’t allotted enough time for testing, the scope is poorly defined and constantly changing or there is a communication gap between your team and management.

Despite the cause, at some point all developers experience a roadblock which leads them to believe their project will almost certainly fail. Scott Ambler, senior consultant on software development and agile adoption, says that software development projects often fail because of an organization’s unrealistic goals for:

• “Scope (what must be built)
• Schedule (when it must be built by)
• Resources (how much it must cost)

The development team has failed at renegotiating the situation, and is forced to try to deliver under those constraints anyway.  In the end, if the project team delivers at all, the quality of the delivered product suffers and the project is almost always late and over budget anyway.”

But when you know a project is failing, can you salvage it before it’s too late? A recent post by ITWorld’s Phil Johnson, covers advice for developers whose project seems to be “circling the drain” so to speak:

“1. Share your concerns with the powers that be

Communicate your concerns in the most concise and non-confrontational way possible up the management ladder. Summarize the risks, but do not try to impose your conclusion on them.
MrFox

Only include facts and objective observations. Leave all conclusions up to whomever (if anyone) reads what you’ve written.
Dan Pichelman

Make a few polite suggestions for improvement. Don’t sound the warning bells, don’t be doom and gloom, just be polite and subtle.
cadmium

2. Continue to do your best on the project 

Try to provide answers, not just problems. Look like you are trying to fix them.
Ozz

Focus on personal excellence – strive to write ever better code, meet ever higher standards of quality and functionality.
code4life

Definitely don’t slack.
Philipp

3. Focus on what can you learn or gain from the situation

Oftentimes, the same chaos and disorganization that comes with a failing project, affords you the opportunity to shine. So look at the project this way: what opportunity does this failing project afford, for the light to shine on all of my strengths and best qualities? What lessons am I learning from this experience, that will make me a better professional and a better person?
code4life

This is your chance to practice important skills in systematic thinking and interpersonal communication. Understand and visualize the issues and potential opportunities that are being missed so you can develop a strategy to communicate these as clearly and simply as possible.
Tone

You can show some leadership skills here that might help you later along the way, treat the problem as a challenge.
Eran Medan

Read more…

Although we don’t play favorites when it comes to dev methodologies, you can’t deny the benefits of Agile development.

The ever-popular approach has helped countless companies increase flexibility within their organization, and thus develop better, more intuitive software.

However, there are ways to do an Agile project all wrong. David Taber, of CIO, recently published a list of the top ways to blow up an Agile project, which he says can make even the best Agile team melt down.

So what are the ‘worst practices’ for Agile development? Here’s a look:

1)      Use The Element of Surprise

“The agile effect comes from trust (through a team that’s deputized and empowered to make key decisions on its own), as well as close communication and relentless focus on the highest-priority goals. Surprise the team with new mandates, unstated top priorities, an uncertain budget, political cross-fire or just a simple lack of trust and it’s hard to imagine anything more damaging to both morale and productivity. Surprise your team and watch it melt.”

2)      Don’t Put Your Best Players on the Team

“One thing’s for sure: The folks who conspired to write The Agile Manifesto are smarter than the average bear. Way smarter. Agile tends to work best when you have users who are articulate and know their business processes cold. Effective agile developers are usually wicked smart.

t’s not just soloist brain-power, though. The team members must respect each other. If they don’t, communication will not be quick and smooth. In my experience, location matters, too. It’s best if team members are within shouting distance, if not in the same room, while it’s almost impossible to have great productivity if the team is spread across multiple time zones.”

3)      Use Politics for Leverage

“You know the drill. Find and emphasize Catch-22 situations, exploit optics and rhetoric, use budgetary threats, shuffle the team, promote and demote people randomly, install a Czar—preferably someone who knows nothing about software but is an I’m-in-charge-here, get-it-done, take-no-prisoners kind of guy—then lather, rinse and repeat. These steps work wonders to show your power, and to break down internal trust and external communication.”

4)      Don’t Give Team Access to Test Resources Until Beta

“What could be duller than testing? Seriously, the continuous building and changing of agile software makes testing an endless chore for the users and a bore for the compliance guys. Besides, the test labs can’t afford to be available all the time.

You’ll hear these kinds of arguments. You have to overrule them. You don’t have to be a card-carrying member of the test-first crowd to know the economics of software defects: Catch them early and fix them cheaply. Here’s the agile answer: If you test throughout the project, you won’t need a beta at all. The software will just migrate into production when it’s ready.

5)      Accel erator-Brake, Accelerator-Brake

“Agile lets developers and users innovate fast and become high-functioning teams. This depends upon continuity of effort and objectives. Sure, detailed priorities shift continuously—but if the global purpose of a project jumps around, the good effects of agile will never take hold.

The same is true if there are interruptions of more than a couple of weeks, as the flywheel never really gets going. Remember: Continuity matters. Due to a number of human factors, restarting an Agile project that’s been stalled for a month is almost worse than starting from scratch.”

Read the rest here>>

As Taber highlights in worst practice #4, beta testing isn’t the right time for testing… and overlooking QA is a sure way to demolish your Agile project. In fact, if testing is done correctly (beginning early on, and done continuously) a beta test won’t be needed at all.

In an Agile setting testing needs to be incorporated into each step of the process, not just at the end of each major release. Real world QA done throughout each step of development, paired with the lab testing that Taber mentions, can help dev teams avoid costly bugs and mistakes. It also keeps the bugs out of the hands of the application’s users, unlike a beta test.  After all, you don’t want to expose your user base to a buggy product. But as you can see from Taber’s mistakes to avoid, the challenges don’t end with testing. Planning for Agile, documenting and organizing Scrum meetings quickly while maintaining an Agile environment requires a unique approach where all the key players are involved and on-board.

Agile development has great potential to help teams quickly and efficiently produce higher quality software – but it has to be done right.

For more best practices on adopting Agile, download this free whitepaper: Ten Tips for Understanding Agile Development and Making it Work for You.

What do you think is the biggest challenges in adopting an Agile development approach? Let us know in the comments section.

Our Testing the Limits guest this month is Jonathan Kohl, a consultant and technical leader who writes and speaks on a wide range of software development and testing topics.

In this interview we talk to Jonathan about his passion for the field, what’s changed over time, his take on mobile app testing and his advice for new and seasoned testers. To learn more about Jonathan visit his website or follow him on Twitter @jonathan_kohl.

********

uTest: I think it’s safe to say that you’ve jumped into the world of software development and testing whole-heartedly. What drew you to this field and how do you stay passionate about it?

JK: The bottom line: I love to build things, and creating great software with a talented team is incredibly rewarding. Knowing that we have created something that helps make people’s lives easier is gratifying. I have found an ideal mix of problem solving, technology, creativity and satisfying and impressing customers in the software development industry.

I have great friends in the industry, and we keep each other sharp, and work on side projects together. We can talk about the latest technology, tease each other about programming tools we like and dislike, or branch off into just about any topic when we’re together. We also serve as a great support network for each other when someone is struggling, either technically, or personally. When we are working together to solve a difficult problem, nothing describes that energy of collaborating and working past your weaknesses, and that triumph of shipping working software at the end.

I ended up in this field by accident – I got lost on my way to law school. In the mid 90’s, I had three university professors who were incredibly influential: Dr. Michael Kubara (Philosophy), Dr. John Rutland (Management) and Dr. David Cowan (Mathematics). I was taking logic and philosophy classes from Dr. Kubara, and he told me to talk to a Math professor because I was delving into territory that he didn’t have expertise in. Dr. Cowan was teaching me Linear Algebra at the time, and jokingly said: “What are you doing with this ancient logic when the real research is in computer science?” Dr. Rutland had us study Data General for his class, and I was reading Tracy Kidder’s book The Soul of a New Machine.  The quirky nature of the technical people in the book appealed to me (I was playing in a band at the time, and read legal journals and philosophy papers in my spare time, yes, I was pretty quirky myself) and I loved the elegance of logic, and the questioning nature of philosophy. I agreed to try out a computer science class, and ended up getting sucked into an internship at a software company not too long after learning to code.

The software company reminded me of what I had read in Kidder’s book Soul of a New Machine, particularly the interesting, intelligent technical people. I enjoyed hanging out with them, and working on really difficult problems. I loved the fast paced, work-hard, play-hard culture of a software startup. I’ve stayed here ever since, with a brief trip back to university to finish off my undergrad degree. Once in a while I am tempted to pursue the law degree again, but I am invariably sucked back into a great project or opportunity.

uTest: You were an early adopter of Agile and later started talking about “post-Agilism.” What are your thoughts on the Agile movement over the years? (Since you could probably write an entire book on this topic give us the dust jacket version.)

JK: I started looking into Agile methods when I was working on a team in 1999 that was trying a combination of Rapid Application Development (RAD), Open Source, and iterative/incremental models like Spiral. The lead developer sent us this article called “Chrysler Goes to Extremes” which described the very basics of Extreme Programming. We liked that it was open and free (we didn’t have to purchase some tool chain and coaching we couldn’t afford) and we started trying some of the ideas out. We then bought the book “Extreme Programming Explained” by Kent Beck, and started implementing XP as best we could. A year later I was introduced to Scrum, and we had good success with it. After a while, I started noticing failures on Agile projects, and people moving on and doing other things. That was fascinating to me. Sadly, instead of learning why there were failures, there was an overwhelming urge to suppress them, or worse, dehumanize people by blaming them for doing it wrong.

As for Agile methods in general, I am ambivalent.  I am glad that lightweight methodologies are much more common place than they were in the ‘90s, and we have benefited from creating a common language around practices, and our tools are so much better now than they were ten years ago. I enjoy working on many Agile projects, since they fit my process and personality, and how I like to work.

However, there is a lot of snake oil out there, with proponents claiming that merely adopting Agile methods will lead to a successful business (What about having a great product, great customer service, skilled people and strong financials? Don’t those sort of matter too?) That really turns me off.  Furthermore, for years, any Agile failure would inevitably involve blaming the victim: you did it wrong, you don’t get Agile, if you were really Agile it would have been a success, which often sounds like another variation on the “no true Scotsman” argument fallacy. Sure, sometimes people fail because they made some mistakes, or didn’t commit, but every Agile project that fails can’t be blamed on the people.

Read more…

Over the years, we’ve thrown a considerable amount of nouns, verbs and adjectives at the topic of development methodologies. We’ve focused mainly on that of agile and waterfall, as they are not only the most popular, but they are (in my opinion) the two most polarizing approaches.

We’ve examined and listened to arguments in favor of one versus the another – agile is more fluid and responsive, waterfall is more practical, agile people are flakes, waterfall people are dinosaurs, etc. etc. etc. We tried to be neutral, giving equal weight to the arguments of both sides. Our aim was to educate you – dear reader – in the hopes that we could help you make the right decision when it came to choosing a methodology.

What a waste of time that was! Well, only if you believe that the war of methodologies are over - an idea that was raised in a recent article on PCadvisor.com, and one that I felt was worthy of further consideration. Here’s the gist:

The wars over development methodologies — agile, XP (extreme programming), waterfall, and so on — are  fast giving way to a more fluid and flexible approach to producing and refining  a product. Telerik’s Semeniuk is one of many in the modern development world who  sees development methodologies not as dogmas to be followed to the letter, but  toolkits to be raided for what’s useful. Confining a development team to one methodology is becoming a thing of the past.

In other words, if you are a methodology fundamentalist – that is to say you never waver from the playbook – then you are likely missing out on a number of tactics that can improve the development (and testing) process. Case in point:

Some, however, caution that agile can’t simply be sprayed onto an existing  development process. A former program manager who has declined to be named but has five years of experience as a Scrum master has time and again seen agile used in development, but with no corresponding changes in other facets of  bringing software to market.

“There’s no intermittent QA; instead, there’s old-school ‘toss it over the  wall to QA’-style QA,” he says. “Instead of regular releases, they’re using agile to get a release out, then having the schedule disrupted by support.” In  his purview, there has been a battle between traditional software releases and agile, with a lot of people simply using agile merely to drive old-school models.

So what now? On the one hand, you can continue to stick to the rigid rules of the methodology of your choice. Or you can accept the notion that every methodology has something of value, and to borrow those elements as they are needed.

What are your thoughts? Do you believe the methodology wars are coming to an end?

Please sound off in the comments section below.

Imagine that you stumbled upon a casino slot machine containing a software bug like no other. A glitch that could make you rich in minutes. Would you pull the lever and take the money? If you did, is that a crime?

This is what federal courts are debating after a Las Vegas local named John Kane uncovered a firmware bug present in 10 casino machines. The glitch had been hidden for seven years.

The casino, known as the Silverton Casino Lodge, housed these faulty machines and had been suspicious of Kane for quite some time. On his last day reaping the benefits of the faulty software, Wired’s Kevin Poulsen, says Kane scored some big wins. In about one hour he scored five jackpots, and after Kane didn’t collect his last win of $8,200, the Gaming Control Board removed the machine and took it to the lab for testing. According to Poulsen:

“Now Kane and the bug he exploited are at the center of a high-stakes legal battle before a federal judge in Las Vegas. The question: was it a criminal violation of federal anti-hacking law for Kane and a friend to knowingly take advantage of the glitch to the tune of at least half-a-million dollars? Prosecutors say it was. But in a win for the defense, a federal magistrate found last fall that the Computer Fraud and Abuse Act doesn’t apply, and recommended the hacking charge be dismissed. The issue is now being argued in front of U.S. District Court Judge Miranda Du, who’s likely to rule this month.”

This incident has to be having a major effect on slot machine-makers worldwide. How many more of these real world bugs exist? Can the Gaming Control Board get to them before users do?

With any software comes the usual bugs, and as developers increasingly use higher technology for slot machines, the bugs themselves are becoming more advanced. Cases like this remind us of the immense value that real world testing provides. It’s near impossible to replicate a real world software bug inside the lab, and these are bugs that users can and will stumble upon if developers don’t catch them first.

Slot machine makers need to reconsider their current QA processes; Who is testing their machines?  If developers are testing their own work – or worse, if they aren’t doing any QA – they need to make changes fast.

What do you think of John Kane’s case? Is it a crime to discover a slot machine bug and make money off of it? Share your opinion in the comments section.

In the software development world – the pressure is on.

Management teams want software developed yesterday, and developers are struggling to keep their heads above water. The push towards agile methodologies and faster development has falsely suggested that quality assurance would soon become the developer’s responsibility, thus reducing downtime and speeding up the dev process. The assumption being that development teams would be the ‘keepers’ of quality, testing their own work. After all, this could save organizations a lot of time and money… right?

Wrong. The cost of testing is minor compared to the cost you’ll pay fixing critical bugs after release, handling customer complaints, pushing a patch and loosing revenue. It won’t save time either. All the bugs and glitches that are overlooked by developers will need to be handled, patched, re-released – and who knows if any of your software’s users will have stuck around when your done cleaning up the mess. Overlooking quality can double, even triple, your development time.

So will QA ever become the developer’s job?  As the SD Times Editorial Board says, “Sorry, that’s not going to happen.” According to the SD Times, quality software will always require quality testing:

“Sure, dev teams can and should focus on building quality into the product. Of course, code repositories can enforce standards for unit testing. Certainly, business stakeholders should regularly review the software to ensure that it meets their business requirements, just as architects should review against technical requirements.

All of those trends will improve software quality by identifying defects sooner so they can be fixed less expensively. All of those trends will shorten software release cycles and lower software costs.

However, no matter how you slice it, sophisticated software will require sophisticated testing that goes beyond what developers can do.

…someone, somewhere, needs to perform QA prior to release. All the automation in the world can’t eliminate the need for QA. Thank heavens.”

The SD Times Editorial Board is spot on  – nothing can overshadow the importance of in-the-wild testing. As a recent article on The Codist Blog says, “No matter how careful or artful a programmer might be about small testing the need for big testing is not diminished. Quality applications are built by a combination of skilled people and one of those needed skills is testing.”

In short, real world QA isn’t going anywhere. Software is so entrenched in consumers’ lives that they have little to no tolerance for buggy software and won’t hesitate to abandon your application if it hasn’t been properly tested. That’s a risk organizations can not afford to take.

Your development deadline is approaching, and you know your work won’t be done on time… or at least done well on time. You have two options; deliver poor quality software and meet your deadline, or miss your deadline all together and suffer the consequences. Does this sound familiar?

In most companies today there is massive disconnect between management and development teams around setting and maintaining deadlines. On the one hand management needs the work done fast, and on the other dev teams need to produce quality, which doesn’t exactly happen overnight.

CIO’s Matthew Heusser recently posted “How to Deal With Software Development Pressure” featuring a group of programmers swapping their development “war stories”.  Some of the programmers include Nayan Hajratwala, principal of Chikli Consulting, Matt Barcomb, agile coach and consultant, David Hoppe and Michael Drzewiecki, a project manager from South Bend, Ind. Here’s a look:

“Hajratwala: These people…have been sold so much snake oil that they have no faith in anything coming to them. So they say, “You know what, here’s what I need and here’s when I need it.” The only way to break through that is to actually deliver something.

What we’re saying—that we can have something in production by the end of the week—makes no sense to them. They want to see the three-year plan. Frankly, until we’ve delivered, they have no reason to believe it.

Keller: Some people have the thought that they get the commitment to November in order to get it in May. The false-commitment makes sure they get it at all, and they believe it makes the team work really hard.

Drzewiecki: That’s schedule pressure as a tool—from a belief that if you don’t apply the pressure, the technical team will be lazy. There’s a lot of distrust there.

Kaufman: Sometimes the hidden message is to deliver the crappiest piece of software as soon as possible.

Keller: Team members worry about a death march. A few people just can’t work longer hours for family reasons. The team often figures out [it] can get it done by sacrificing quality.”

While each scenario varies from company to company – sacrificing quality to meet a deadline is more common among development teams than you might think. Bridging the communication gap between business leaders and developers is the first step towards eliminating this problem. Sketchboards, storyboards, flowcharts or wireframes are a few of the ways to facilitate better, more effective communication.

However, most importantly teams need to make quality their top priority.  Under pressure developers chop off certain features, and shorten their time for testing. Releasing a buggy product to consumers puts brands at risk of ruining their reputation. Software has to work, every time.  Extending your QA and testing in real world scenarios can help bring quality back to your applications.

We want to hear from developers; how do you deal with the pressure of software development deadlines? Let us know in the comments section.

Software is no longer a route, an option or an afterthought.

Today, software is the basis for just about every business and institution out there. Unfortunately, as the number of computer programming jobs grows at 2x the national average, the number of students being educated falls short.

According to an infographic from code.org, there will be 1,000,000 more computer programming jobs than students by 2020. And while it is among one of the highest paying career fields, less than 2.4% of college students will graduate with a degree in computer science.

Is this unbalanced ratio caused by students’ lack of interest, or is it an issue within the education system?

Code.org adds that 9 out of 10 schools don’t even offer programming classes. However, there are several programs available that set out to provide people with education in programming and software development. The successful initiatives today include code.org; codeacademy.com; Code School; Khan Academy; Code Avengers; Programr; Try Ruby.

The president has addressed this imbalance. As mentioned here earlier in February, the President stressed a need to focus on tech education in his State of the Union address. There have also been several high schools and after-school programs that have recognized the need, and are joining the tech movement.

While the movement is underway, Ole Lensmar of NetworkWorld says there is an important piece missing from this initiative – testing:

“If anything is missing from this initiative, in my opinion, it is a focus on the sister discipline of software testing. It’s an area that is often underrated by some organizations, that is, until they try to use a buggy tool or an unstable API. If we are going to rely so heavily on software as the infrastructure for the finance, medical, government, and education sectors, it’s important that we build quality software those industries can rely on. I would prefer that my doctor have bug-free access to my medical records and that my banking transactions get posted accurately. And, with all due respect to programmers, I like to think that those applications are thoroughly inspected by professional testers before they hit production.”

Medical records and bank transactions are only a few of the ways we use software to access our private data. This is why testing is so essential across all software verticals. Lensmar adds that while test automation is great, nothing can replace true, in-the-wild testing:

“Software testing has been on a path toward convergence with programming for some time, especially when it comes to test automation. At the same time, many vendors have tried to create tools that allow non-programmers to exercise code in ways they would not know how to do by themselves. It’s surprising how slowly this discipline has evolved over the last two decades – often because it has all the overhead of code maintenance without any associated direct revenue. And yet, the benefits of having well-maintained automated tests in a continuous integration environment can’t be understated. At the same time, software testers provide much-needed input and perspective from exploratory testing and user acceptance testing. In my opinion, nothing can replace the “human perspective” as part of the quality assurance cycle.”

The human perspective is an essential part of the QA cycle, and Lensmar is right, testing cannot be overlooked or shorted. The QA process is just as, if not more, important than the actual development process. Testing under real world human scenarios cannot be replicated through an automated tool – or even inside the lab. However, there are solutions available today that have access to experienced, educated testers. With the right crowdsourced in-the-wild testing tool, companies and programmers can gain access to the testing talent they need and get the human “real world” perspective their apps require.