Software Testing Blog

Having just finished Steve Jobs biography, and being of the school of gated platforms – at least for my phone, where I don’t want to deal with bugs the way I might in my work laptop (sorry Matt B and the uTest IT team) – I found this concept very interesting.

According to the BI article, “Android Hackers Plan App Store of Banned Apps,” a group of Android Developers are looking to start their own app store for all the banned and rejected apps that didn’t make the cut.  The article includes a quote from the potential founder that, “apps removed from the Market include, one-click root apps, emulators, tether apps, Visual Voicemail apps, and more.”

It sounds great but we already know about the growing number of malware on phone operating systems, the Android especially.  The other alternative for apps is to create mobile-specific landing pages (i.e. HTML5 apps), like Grooveshark (music) and Untappd (beer reviews) have done, making the apps available via your mobile browser.  Since their launch, Untappd has launched a native app for iOS and Android but has not shared details on traffic comparisons.  [It won’t be applicable to most mobile users but we cover some security exploits and common attacks in our Security Testing whitepaper.]

Am I the only one uber-sensitive about the integrity of my phones OS and Apps?  Would you download an app that isn’t scrutinized for security?

Need to find a nearby restaurant? There’s an app for that. Want to track your workout progress? There’s an app for that. Want to play a game to kill the time? There’s an app for that. Your kids want to play a game? There’s an app for that. Want to check the score? There’s an app for that. Want the latest headlines/facebook status updates/tweets? There’s an app for that. Want to access that document or program you use at work on the go? Maybe there’s an app for that, kind of.

Many consumer apps have figured out how to cater to the specific limitations of mobile devices (screen size variance, touch screen usability, the range of OS options) but professional apps are still largely lagging behind.

Quinton Alsbury (who owns a company – MeLLmo – that develops apps for businesses) highlighted where many business-minded apps are going wrong in this guest post on CNet:

We’ve all been there–squinting at a spreadsheet on a mobile device, zooming in and out in attempt to make sense of the information. Each swipe of the finger triggers a blank screen as the data renders and slowly reappears and our frustration builds. …

Why do apps aimed at business users continue to cram features and functionalities designed for the PC into a mobile phone, ignoring all the things that make consumer apps successful–namely, design, speed, and interactivity?

Many business app developers are fundamentally misunderstanding the mobile user experience by producing “shrink to fit” versions of solutions designed for the PC. The mobile experience isn’t about accessing several gigabytes of data; it’s about quickly accessing the information you need, when you need it.

By “shrinking” existing PC tools, they’re essentially jamming a large, complicated, and bulky system onto a smaller screen. What results are apps that contain too many features, respond too slowly and ultimately result in low user adoption and usage.

Read more…

‘Tis the season to prognosticate.

We’re 17 days away from the new year, and far before Auld Lang Syne begins playing and we pretend to know the words (after all the champagne, who can remember the lyrics we optimistically Google’d the day before anyways?), we’re pondering what changes are in store for us the next twelve months.

In a whitepaper released by ABI Research this week, their tech analysts took a collective look into the crystal ball for 2012 and (in their words) “have drawn some bold lines in the sand on a plethora of top-of-mind topics.”

But instead of predicting what WOULD happen in the mobile and telecom space, they took a different spin on the usual list and forecasted what WOULDN’T happen.  Nice twist.  (And a really good read.)

One of their more interesting predictions for those of us in software testing is by Patrick Connolly, Senior Analyst of Telematics and Navigation:  “Indoor location will NOT become commonplace in 2012.” 

It’s easy to see how this could be true…but also surprising.

After all, for as many articles that have been written about the technological challenges in making Indoor Location Based Services (LBS) a reality, there has been an equal amount of big name, big buzz announcements about it over the past few months.  There are dozens of industry-leading companies—including Apple, Navteq, Qualcomm and Nokia—tackling the challenge from every angle.

There are even some major apps launching to give Indoor LBS a jolt from vision to reality.  For instance, Google announced on their Mobile blog in November that the new Google Maps 6.0 gives users (on Android OS 2.1 mobile devices) the ability to Map the Vast Indoors, vis-à-vis:

Read more…

Here at uTest, we have the unique privilege of seeing some of the world’s latest, greatest mobile apps before they make it big. After years of testing these apps, we’ve also gotten pretty good at spotting the likely winners and losers through several distinguishing characteristics. These include functionality, security, ease-of-use and most importantly: originality.

As you’re probably aware, not every app is original. In fact, most of the apps in existence are anything but unique. Scroll through your app store for five minutes and see if you can argue otherwise.

Anyway, this glut of apps – especially copycat apps – has gotten so ridiculous that it’s got several high-profile tech bloggers essentially begging to make it stop. Here’s an excerpt from Can We Stop The Copycat Apps from Rip Empson of TechCrunch:

While I encourage developers to continue making great apps, I do question the need for both making and for approving the parade of — for lack of a better word — “rip-off” apps. What am I talking about? Example: Over the last week, I’ve watched another fairly blatant copy of Angry Birds hover inside the “Top Free iPhone Apps” list on the App Store, even grabbing the second spot at one point.

I’m not naming the app explicitly, because I don’t want to give the game free publicity. That’s what they want, and it’s probably a good idea to avoid promoting the production and downloading of spammy (cr)apps. But needless to say, the scenario is familiar: The game’s icon is practically identical to that of Angry Birds, it has “Angry” in the title, the design and gameplay — while not exactly identical — have enough similarities to make for some serious eye-rolling. Not to mention, the game is awful. One-star reviews abound.

Read more…

Last week Matt Solar posted about a 6th grade mobile app developer, so let’s continue that youngster trend. Ever since computers started appearing in homes there have been young people tinkering with technology and creating amazing things. Now Winchester High School in Winchester, MA is extending that young, tech-savvy, creative spirit from the home/basement/parent’s garage into the classroom. WHS is now offering a “Designing Applications for Android” course. Here’s Winchester Patch with the details (emphasis added):

Four-year  WHS Technology Teacher Daniel Downs, realized that the future of technology is moving toward mobile devices. He has created a curriculum that challenges the students in the design, implementation, and testing process using tools chosen for their superior interactive educational value. These tools such as Adobe Flash CS5 and Accelerometer programs, allow the students to achieve an unusual classroom success—the success of being able to design, implement, test, and immediately use their designs on technology already part of daily life. …

The Designing Applications for Androids class, in two months, has designed and published 32 original Children’s Games. … These are all currently published on the class’s tablet computers. Currently students are designing Apps to potentially replace the school’s paper agenda planners. …

Soon, students will conduct a survey of the entire WHS community’s mobile device application needs. Then, they will design apps to satisfy these needs. In one semester, students will have learned to design, implement, test, survey, and design for others.

Read the whole article >>>

Read more…

Ho, ho, ho!  Whoa there, Blitzen– wasn’t it just Halloween?  It sure feels that way. After all, I still have two pounds of trick-or-treat candy to pretend I’m not eating.

Unfortunately, my four-year-old has already implored me to take down the skeleton and spiders hanging in the doorway because they’re going to scare away Santa.  So, rather than arguing the salient fact that Santa shimmies down the chimney versus ringing the doorbell, I’ve officially started morphing decor from the marvelous macabre to merry old Saint Nick.  Kids: 1. Mom: 0.

Nonetheless, the fact hasn’t escaped me that we’re two weeks away from Cyber Monday (November 28th), an occasion that online retailers have been planning for months.  Since summer, global brands and independent e-tailers have been testing and re-testing their mobile apps and web sites for functionality, usability, localization glitches and possible bottlenecks in site performance that could jeopardize their revenue potential.

Moreover, the ante has been upped now that the iPad and other tablets have entered the scene.  Online retailers that spent the last few years optimizing their mobile apps and porting them to additional platforms like Android, are now going through the process from scratch with tablets.  Not only are the specs non- standardized, varying significantly by manufacturer, device and network performance like smartphones.

Read more…

Our Testing the Limits “reunion tour” rolls on this month with Michael Bolton, back for another lively session of Q&A. Michael is best known as the founder of DevelopSense, his Toronto-based testing consulting firm, and as a leading figure in Rapid Testing and the Context-Driven school of testing. In short, he’s one of the industry’s most highly regarded writers, speakers and teachers – and it’s a real pleasure to have him back. For more on Michael, be sure to check out his website, blog or follow him on Twitter.

In part I of our healthy two-part interview, we get his thoughts on test cases not being related to testing; the sub-par debate skills of testers; the quality chain of command; objections to Rapid Testing and much more. Be sure to check back tomorrow for Part II. Enjoy!

uTest: It’s been almost two years since our last interview. Where does the time go? We’ve followed you pretty closely during that time (on Twitter, don’t worry), but for those who haven’t, what have they missed? New publications? New courses? New ideas on testing? What’s new with Michael Bolton?

MB: I’ve been traveling like crazy this year, and I’m booked pretty heavily through the end of the year.  I’m beginning to set up my schedule for next year—so if people would like to schedule an in-house class, now is a great time to ask.  For new publications How to Reduce the Cost of Testing, a new book edited by Matt Heusser and Govind Kulkarni, has just been released. I’m pleased to say that I’ve got a chapter in there, with a number of other members of our community.

I don’t specialize in new ideas in testing so much, but rather in refining and reframing ideas we’ve had for years in more specific and, I hope, more useful ways. The other thing that I love to do is to bring ideas from elsewhere into testing.  Currently I’m fascinated by the work of Harry Collins, who studies the sociology of science and the ways in which people develop knowledge and skill. Tacit and Explicit Knowledge is his most recent book; The Shape of Actions is older.  I’m most interested in the idea of repair, which is Collins’ notion for the ways in which people fix up information as they prepare to send it, or as they receive and interpret it.

As an example, I’m 5’ 8” tall.  If I ask you how tall I am in centimeters (and provide you with the ratio of 2.54 centimeters to the inch), you’ll probably do a little math in your head to translate 5’ 8” into 68 inches.  If you do that, it’s because you have tacit knowledge that a foot is 12 inches, and it’s quicker to do five times 12 in your head and add eight than to work it out on the calculator.  Then you’ll report that I’m 173 centimeters (or 172), rather than what the calculator tells you:  172.72.  If you round the answer up or down to a whole centimeter, it’s because you have tacit knowledge that the extra precision is useless when my height changes more than that with every breath. The calculator doesn’t know that, but people often fix up the interaction with the tool, applying that kind of tacit knowledge without noticing that they’re doing it.  Collins argues that we give calculators and computers and machines more credit than they deserve when we ascribe intelligence or knowledge to them, even when we do it casually or informally.

My latest hobby horse is definitely not new, but I’d like to have a go at it anyway.  I’d like to skewer the idea of the test case having any serious relationship to testing.  Test cases are typically examples of what the product should do. That’s important; we often need examples to help explicate requirements and desires. But examples are not tests, so I’d like to call those artifacts example cases or examples rather than test cases. They’re confirmatory, not exploratory; checks, not tests. Brian Marick has written a lot about examples; Matt Heusser has too; so has Gojko Adzic. James Bach has been railing about test cases for a long time.  Often test cases are overly elaborate, expensive to prepare and maintain.  They’d be even more expensive if testers didn’t repair them on the fly, inserting subtle variations making observations that the test case doesn’t specify.  Just as Collins suggests about machines, test cases get more credit than they deserve.  As Pradeep Soundarajan would say, the test case doesn’t find the bug.  The tester finds the bug, and the test case has a role in that.  Now: the development of checks and the interpretation of checks—those things require all kinds of sapience and skill.

A test, to me, is an investigation, not a bit of input and output for a function.  Yet people tend to think of testing in terms of test cases.  Even worse, people count test cases; and even worse than that, they count passing and failing test cases to measure the completeness of project work or testing work.  It’s like evaluating the quality of a newspaper by counting the number of stories in it without reference to the content, the quality of the writing, the quality of the investigation, the relevance of the report, whether a given article contains one story or a dozen, and so forth.  Counting stories would be a ludicrous way of measuring either the quality of the newspaper or the state of the world. Yet, it seems to me, many development and testing organizations try to observe and evaluate testing in this completely shallow and ridiculous way. They do that because seem to think about things in terms of units of production. Learning, discoveries, threats to value, management responses… none of these things are widgets. They not things, either, for that matter.

uTest: In a recent blog post, you wrote about the inability of some testers to properly frame tests, mainly because they haven’t been asked to. Generally speaking, what other qualities or skills do you find testers to be lacking in?

Read more…

I’m talking about the Android Ice Cream Sandwich (ICS) – the fourth major Android OS version – which is growing closer to its release! Google is urging developers and testers alike to get ready for it, so consider yourselves forewarned. For now, what’s most important is to make sure your apps work on large screens AND small screens as this “cool” release is going to run on both tablets and smartphones.

According to CNET:

“Developers who created their apps specifically to run on Honeycomb-based tablets will need to tweak their APKs (Android packages) to either prevent or support their installation on smaller-screen devices.

The [Google Android developers] blog also offered some recommendations for tablet app developers on how to ensure that their design of the Action Bar widget works on smaller handsets.”

Read more…

As you may have read on Monday’s blog post, uTest launched a new informational campaign to promote http://www.inthewildtesting.com.   The web site – and associated social media channels, including a Twitter profile – are intended to educate forward-thinking technology leaders about the necessity, benefits and real use cases of in-the-wild testing. 

We decided to launch it at TechCrunch Disrupt in San Francisco because the very concept of in-the-wild software testing (versus traditional methodologies) is, well…disruptive. 

Sure enough, TechCrunch Disrupt turned out to be the perfect event!  There were more than 2,600 innovative, entrepreneurial-minded techies, investors and exhibitors (35% more attendees than expected) filling the halls of the Design Concourse Center from Monday to Wednesday.  In its usual fashion, the conference itself attracted top industry-leaders such as Reid Hoffman of LinkedIn, Marissa Mayer of Google, Vinod Khosla, and even Ashton Kutcher.

uTest hosted a ton of terrific activities over the course of the event:

Read more…

Every few weeks, it seems like there’s another major security breach to the website, gaming system or native app of a big global brand.  And that doesn’t even include the hundreds (thousands?) of hacks into the properties of smaller enterprises, SMBs and startups that consumers may (or may not) hear about.

In fact, a few months ago we wrote about The Top Security Hacks of 2011, and referenced that the attacks on Playstation were estimated to have cost Sony $24 billion dollars– nearly 10x their revenue for the same period.

So here’s the point: Would you rather look back and say your company overshot and used too many systems for security testing?  Or get that nauseaus, sinking feeling in your gut when your CIO wakes you at 2:00am and says the company has spent too little?

That’s why– as the cornerstone of uTest’s showstopping announcement yesterday– we announced the launch of uTest Security Testing that leverages the talents of new and existing white hat security professionals within our crowdsourced community.  Since we now offer the first crowdsourced, real-world security testing in the world…there’s a new kid in town to join the collective effort to protect your company, and customers’, private data.

Moreover, we’ve joined forces with industry leader Veracode to provide seamless access to their complementary, cloud-based application security verification services.  Veracode has scalable, policy-driven application risk management programs that help identify and eradicate numerous vulnerabilities by leveraging best-in-class technologies from vulnerability scanning to penetration testing and static code analysis.

As a result, companies will have access to a cost-effective, powerful combination of automated (Veracode) and real-world (uTest) testing that mitigates security risks across the entire software development lifecycle.

We’re thrilled, honored and excited to be partnering with Veracode.  And we’re certain that our joint offering– as a complement to organizations’ in-house security testing– will offer tech executives peace-of-mind at a price with infinitely fewer zeroes than $24,000,000,000.