Testing the Limits With SOASTA’s Dan Bartow – Part II

In part II of our interview with SOASTA’s Dan Bartow, we get his thoughts on why load testing often gets neglected; advice for test tool selection; the challenge of load testing for mobile apps; SOASTA’s plans for 2011; fights between flying sharks vs. flying crocodiles and more. If you missed our previous segment, you can read Part I here. Enjoy!

uTest: True or false: Load and performance testing is often one of the most neglected phases of software quality. Please explain why this is (or is not) the case.

DB: True!  The time allotted for QA in the software development lifecycle has always been the first thing to get squeezed when a project gets behind.  Traditional software methodologies such as Waterfall essentially go from requirements to development and ultimately QA at the tail end.  When the project development activities get behind then the only thing left to cut from and still deliver a product on time is the QA cycle.  Performance still isn’t a part of many project plans today (this is almost a separate topic in itself), but when it is in the project plan it usually gets a slice of the QA time which is already too short in most cases.

Now we live in an agile development world and while agile functional QA is catching up we still don’t have agile performance testing as an industry standard.  The reason for this is that the dominant product in lab performance testing, HP LoadRunner, requires you to write code for your performance tests that is more complex than the actual web application code you’re testing.  If you have to write your test cases in C and it takes two weeks to write an end-to-end scenario on a finished web app then you have dead weight in your dev lifecycle.  As a result of these weaknesses companies have lost confidence in the value of performance testing their apps.  The way to reinstate this confidence is with a modern testing tool and a modern approach to testing.

uTest: How important is tool selection when it comes to load and performance testing? Are testing failures a result of this or something else, like personnel?

DB: Tool selection is very important for overall success although testing failures can be because of people, processes and/or technology.  You need the right tool for the job and you need the right people to use them in a process that’s set up for success.  Just like QA isn’t a one size fits all shoe neither is performance testing.  Personally though I think most testing failures are a leadership and execution problem and not because of the tools being used or the processes.  Quality comes from the top down.  The companies out there delivering the highest quality offerings are the ones that build quality in from the CEO all the way through the company.  Probably every tester reading this knows what its like to be a QA Engineer at a company that doesn’t seem to actually care about quality.  How ironic!  I said tool selection was very important, but I really don’t even want to focus on that here because tools are just tools.  Before you worry about whether or not you have the right tools time should be spent on making sure you have the right attitudes on your team and the right players.  Once you have a good enough team that is pushing the capabilities of your toolset then I think you’ve got a foundation for success and you can start driving higher.

uTest: How does the expansion of mobile apps and devices impact load testing? Is this a game-changer? Or something current load testing is well suited for?

Continue Reading

Is 2011 The Year Of The Mobile Enterprise App?

We write a lot about the latest and greatest apps on smartphones, tablets and other devices. But what about enterprise apps? We haven’t taken a look inside the enterprise for signs of a mobile app uptick (support or implementation) in quite sometime. That’s why this study conducted by Kelton Research really struck a chord.

Yesterday, Kelton Research announced the results of their study on mobile enterprise apps, and I think they may shock you. According to Kelton, a whopping 90% of U.S. and UK IT managers surveyed will implement new mobile apps in 2011! 250 IT managers in the U.S. and UK at companies with revenue of $100MM+ were polled.

At first, I thought these stats seemed slightly inflated, but the more research I did, the more surveys I found that indicated that 2011 is indeed the year of the mobile enterprise app. In fact, ABI Research expects worldwide enterprise mobile data revenues to reach $133 billion by 2014.

With new apps for BlackBerry, iPhone, iPad, and Android making deep in-roads in healthcare, finance, education, media, and retail, employees are using mobile apps to access everything from CRM systems, to financial results, to marketing campaigns, to tracking orders, etc.

Another very interesting finding in the Kelton study… Saving money is the most popular reason (63%) for deploying mobile apps (chart above). I agree with Eric Lai of ZDNet who said, “It’s heartening, as it shows that mobilizing appears to be a fiscally sound strategy, not something with a pie-in-the-sky ROI.”

Some other interesting stats from the study include:
Continue Reading

Your Weekend Testing Round Up

I’ve been cyber-bullied by coworkers into writing a blog post. I’ll admit, I’m one of the marketers prime for the blog bribery that Kipp Bodnar speaks about in his recent blog post on convincing coworkers to blog. It’s not that I’m anti-blogging, it’s just that like the TI-82 did to my memory of equations, Twitter has ruined my ability to write in quips longer than 140 characters.

But it’s never too late to get it back (at least that’s what my co-workers told me). So to ease myself back into the process, I’ve decided to summarize a few testing-related stories I recently saw in the news. Here goes….

Mobile App Reviews
Read Write Web recently posted a nice list of mobile app recommendation sites that developers can use to market and sell their apps.  While no single site has the niche-market share for apps that TripAdvisor does for hotel reviews, they’re certainly battling it out quickly. With the Apple Store having over 250,000 apps available to 100MM people, and a staggering 5B app downloads (yes, that’s billion) it’s certainly going to be a race to become the app review site – and one that developers should certainly utilize to their advantage.

Continue Reading

Mobile Barcode Scanning Up 700%, Android Leads The Pack

As 2010 starts winding down, the mobile app revolution continues to wholly define this year in tech. Every day more mobile innovations are being updated and perfected to match our – the mobile consumers – needs. One such emerging trend is mobile barcode scanning.

According to ReadWriteMobile, a new study by barcode tech company ScanBuy claims that barcode scanning is up 700% in 2010!

Android was the most popular smartphone platform by far with 45% of barcode users, followed by Blackberry (27%), iPhone (15%), Symbian (9%), Java (3%) and Windows Mobile (1%).
Other Key Findings Include:

  • Health and beauty products were the most popular items among 1D (UPC) scans with 21% of users, followed by groceries (14.4%), books (12.6%) and kitchen items (9.2%).
  • Over 45 countries have scanned barcodes.
  • Linking to a website is the most popular action delivered by a 2D barcode scan with 85% of scans.
  • 1D (UPC) and 2D (QR) codes are being scanned equally.

In response to this huge news, I thought I’d have some fun with URL shortening service bit.ly‘s new QR-generation tool that launched a few days ago (FYI: goo.gl launched a few weeks ago too). Go ahead! Scan away and see where it takes you! (Hint: I am the PR Maven ;).)

While barcode scanning isn’t new technology by any stretch, the 2010 mobile boom is driving its increasing popularity. Even Calvin Klein recently replaced its massive billboards in New York and LA with QR codes (pictured above) – not to mention the giant QR codes in Times Square! I wonder what new and exciting mobile app testing doors this will open…

Where’s The App For That? Tweet Your Best Ideas and Win A New iPod

Ever visit your app store of choice — whether it be Apple, Android or BlackBerry — and not find the app you really need?

Well, here’s your chance to tell the Twittersphere your best ideas. Be creative, be innovative, be wacky & off the wall; the best ideas will win shiny new iPods.

1. Follow @uTest on Twitter
2. Be creative! Tweet/Reply to @uTest with your wacky or useful, non-existent mobile app with the contest hash tag #KickAssApps

Example: @uTest a video app that watches where you’re driving so that you can text, email, play doodle jump & drive simultaneously #KickAssApps

The contest will run for the duration of this entire week — Sept 20 thru Sept 24 (11:59pm ET) — so get your ideas in soon!

The top ten most creative/entertaining/funniest mobile apps will get shortlisted by the uTest team. We’ll then post the top ten best ideas on the uTest blog for three days — Sept 28 thru Sept 30 (11:59pm ET) — and let our readers choose their favorite app through a poll. The “inventors” of the top three apps with the most votes will win a new iPod! We’ll announce the winners on Fri, Oct 1.

–1st Place: iPod Touch
–2nd Place: iPod Nano
–3rd Place: iPod Shuffle

So, next time you’re on the road — working, gaming, social networking, reading, listening to music, taking photos, or just checking the weather — and can’t find the app you’re looking for… Ask yourselves, “Where’s The App For That?” and tweet your best ideas. Good luck!!

For official contest rules and legalese, see our Terms & Conditions.

Mobile App World, London: October 19-20, 2010

Apps! Apps! And more apps! As the summer starts winding down here at uTest, we’ve been able to take a step back and a closer look at the big trends emerging all around us. What has been most apparent is the tremendous spike in mobile app testing needs. From top marketing agencies to retail giants to social gaming startups, our customers are developing more mobile apps to grow (or define) their businesses than ever before.

According to Game Developer Research, 25% of game developers are now making mobile games – that’s up from a mere 12% in 2009!

In addition, a survey conducted by iGR found that more than half (53%) of US mobile developers are building apps for Apple’s iPhone OS. BlackBerry was the next most popular, followed by Android and Windows Mobile.

In response to this incredible momentum, this year marks the launch of Mobile App World 2010, where global leaders in mobile tech and app development and entrepreneurs will gather to network and learn about the latest developments and innovations.

uTest will be among the outstanding line-up of more than 40 speakers, which includes Google, Microsoft, Ericsson, Orange Global and the BBC, who will be discussing the future of mobile apps. Shoot us a note if you’ll be around!

Note: If you’re looking for some cool, new mobile apps, check out Mobile App World’s August Apps Of The Month. You may spot a uTester’s favorite app! :)

MobileAppTesting.com Debuts — Promises To Tell You What’s What In Mobile

Like a rocket ship breaking the bounds of Earth’s gravity… like a bird soaring majestically over the open sunlit plains… like a spit wad hurled from the back of the classroom… today, uTest announced the launch of MobileAppTesting.com.  You can also follow our wit and wisdom on Twitter @mobile_app_test.

One thing that you won’t find on this site are ads, subscriptions, hooks or any kind of commercial agenda (don’t tell our investors!).  In fact, we created this site simply because mobile is the next frontier of app development and testing – and the fastest-growing segment of uTest’s business. So we wanted to give something back to mobile app developers, testers and entrepreneurs — and have a little fun at the same time.

We’ll will work with partners, pundits and pioneers (actively seeking co-conspirators) to create original, thought-provoking content about the entire mobile app ecosystem — from app developers to device makers to wireless carriers.  Whether it’s the apps arms race, the constant carrier battles, or the next must-have device, MobileAppTesting.com will be there with equal parts education and entertainment.

This site features user-generated content, contests, product reviews and guest interviews with mobile execs… stuff you can’t find anywhere else, including:

  • Hard-earned lessons for entrepreneurs, techies and investors who want to create world-class mobile apps
  • Overcoming obstacles unique to developing and testing apps for iPhone, Android, BlackBerry and Symbian
  • Breaking news and product reviews (for apps, devices, networks and more) from top bloggers and journalists on the front lines of the mobile app explosion
  • Interviews from people who live it, offering their insights from the worlds of mobile app marketing, design, development and testing
  • Following uTest to mobile industry conferences, networking events, speaking opps and meet-ups

Want to be published on MobileAppTesting?  Have a topic you want us to tackle?  Feel the need to ask what the heck we think we’re doing?  Shoot us a note or drop a comment.

Is Apple Taking Over The Mobile World? The Numbers Tell A Different Story

If media coverage equaled market share, then I’d be writing this post from my iPhone (I’m not) and every single one of you would be reading it from your shiny new iPad (you’re not).  In case you haven’t been near a TV… or a computer… or a radio… or people… you’re aware that Apple launched a new product last week called the iPad.

And with the apparent ubiquity of the iPhone, one can only assume that Apple’s mobile market share hovers somewhere between 97% and 109%.  Unless, of course, you look at those pesky “statistics”, which is exactly what the fine folks at Comscore do each month. As  Jason Kincaid (@jasonkincaid) discussed recently, the latest mobile market share stats might surprise you:

Continue Reading

Users Love the iPad (bugs and all)

As expected, the iPad has received some terrific reviews following its debut to US consumers this past weekend. Users were quick to marvel at its sleek look and feel; its media capabilities; its usability and about a million other features.

Tech guru Matt Cutler (@mcutler) said “it makes your phone screen seem really small… and your laptop feel pretty dated.” Engadget writer Joshua Topolsky (@joshuatopolsky) called it a “potentially a prime mover in the world of consumer electronics” and Steven Colbert especially loves the way the iPad makes fresh salsa.

Is there anything it can’t do?

Of course. With the fanfare comes the feedback – both positive and negative – as evidence by the bugs, glitches and other inconsistencies that have since been reported. Here are a few of the more commonly referenced issues:

Continue Reading

Placing Big Bets On Mobile @ CTIA In Vegas

Whenever you put a bunch of big brains with vast expertise about a still-evolving industry in the same room, you’re bound to get some interesting and impassioned debates.  Such was the case at Monday’s pre-conference sessions at CTIA Wireless in Las Vegas. There were a number of excellent speakers representing the perspectives of OEMs like Nokia and LG; carriers like Verizon and AT&T; and content providers like Travelocity and MTV.

I sat in on several of these sessions and heard a number of important topics being discussed that will have major implications for the future of mobile apps and mobile commerce.  These debates included:

  • Android vs. iPhone vs. Blackberry vs. Symbian
  • Free vs. Paid apps
  • OEM app stores vs. Carrier app stores

But perhaps the most interesting, fierce and recurring debate that I heard at CTIA was around the topic of…Continue Reading