Software Testing Blog

The impact of check-in services, like Foursquare, on personal privacy and security is yet again making top headlines. If you remember our most recent bug battle (The Check-In Challenge), more than 80% of respondents responded “Yes” when asked if they were concerned about how location-based services (LBS) could impact their personal privacy and safety. And 49% chose “privacy/security concerns” as the top reason they don’t use check-in services more often.

Yesterday, the security company WebRoot came out with a study discovering similar results. After surveying 1,500+ social network users with geolocation-ready mobile devices, WebRoot found that more than half (55%) of respondents fear the loss of security and privacy, and 45% are very concerned about letting potential burglars know when they’re away from home (ah yes, the now shut down PleaseRobMe experiment comes to mind).

What’s most interesting to us is that 39% of those surveyed by Webroot said they use geolocation services, but take a look at the number of people that have fallen prey to social network cyber-criminals:

• Nearly a quarter of respondents (22.4 percent) were victims of a phishing attempt to steal their social network password.
• About one in six (16 percent) reported a malware infection in the past year that originated from a social networking site.
• One in nine reported at least one of their social network accounts had been compromised or hijacked.

Even in the face of these risks, many consumers admitted to engaging in risky behaviors:

Read more…

Safe to say that mobile app development has greatly outpaced mobile app testing over the last few years. In other words, while the applications and platforms have seen tremendous technological advances (iPhone 4 bugs notwithstanding) the same cannot be said of mobile testing methodologies.

Case in point: The majority of mobile app developers remain overwhelmingly reliant on internal beta testing.

Here with proof is VisionMobile, who recently published a fascinating report on the growing mobile app ecosystem – a must-read for anyone involved in the space (developers, marketers, users, etc). From a QA point of view, the report further establishes that although testing innovations will ALWAYS trail those of development, the gap need not be so wide.

Here’s an excerpt that sums the whole thing up:

Internal beta testing is the most popular technique used by the vast majority (nearly 70 percent) of respondents, with beta testing with users and peer reviewing the next most popular techniques. Only 20 percent of respondents use focus groups or research of their own. Overall, North American developers are somewhat more sophisticated in their application planning, with 97 percent using beta testing as a standard part of application development and with broader use of a portfolio of planning techniques as well.

Yet, small development firms have limited means today to beta test and peer review their applications with a crosssection of representative users. Given the hundreds of thousands of mobile apps, we believe that efficient (crowd-sourced) testing of apps in a global market of users is considerably under-utilized. This presents an opportunity for the few solution providers in this segment – Mob4Hire and uTest.com, for example – but also for network operators, who can generate a channel for testing applications with end users, and provide an open feedback support system back to developers.

Other notable findings included:

Read more…

So maybe it wasn’t AT&T’s fault after all.

Apple recently revealed that there is a fundamental flaw in their method for calculating how many signal bars to display.  And we have the iPhone 4 (and its “learn to hold your phone the right way” fiasco) to thank for bringing this software snafu to light.

CNN Money shares the following details from Apple:

“Upon investigation, we were stunned to find that the formula we use to calculate how many bars of signal strength to display is totally wrong,” Apple wrote in a statement posted on its website. “Our formula, in many instances, mistakenly displays 2 more bars than it should for a given signal strength.”

That means, for example, that iPhones sometimes display four bars when they should be displaying two. Apple said users reporting a significant drop in bars when they hold their iPhone 4 are probably in an area of “very weak signal strength” but were unaware of that because the phone displayed four to five bars.

“Their big drop in bars is because their high bars were never real in the first place,” the company said.

Perhaps most surprising, Apple disclosed that the problem is not confined to the iPhone 4.  The faulty formula has been present in every iPhone model since the 2007 original.  Questions remain about whether the issue is strictly software-related, or if it also involved hardware problems.  However, Apple has said it will release a free software update in the next several weeks to fix the glitch. It will use a new formula recommended by AT&T.

“I would assume whoever was testing these would have noticed these issues,” read one Mashable comment. Of course, this was made in reference to the slew of bizarre problems reported by iPhone 4 users today. Namely, the infamous yellow-screen and “hand-held” reception issue that everyone – and I mean everyone – is talking about.

I understand their frustration, but in fairness to Apple’s testing team, these are certainly NOT normal issues and it’s likely that most users will never encounter them anyway. Yet because the defects are so unusual (and because Apple is sorta popular) it follows that they’re lighting up the blogosphere, and likely causing more than a few sleepless nights.

I’m reminded of the Seinfeld episode where George bought a cashmere sweater for Elaine. The sweater was perfect – just what she had always wanted – with the exception of a small, red-dotted stain. Though barely noticeable, it was enough to cause sit-comic chaos, before Elaine eventually “re-gifted.”

Anyway, we doubt users will “re-gift” their iPhones, but the issues are certainly enough to annoy the average user. Here’s Mashable with the scoop:

Read more…

News has been all over the web the past few days about the AT&T and iPad security breach.  If you haven’t heard the details, in short a group of hackers discovered a vulnerability in AT&T’s private web APIs where one could send the ICC-ID from an iPad SIM card and AT&T’s servers would send back the corresponding owner’s email address – no authentication required. Since the ICC-IDs for the iPad are somewhat predictable, it was trivial for the hackers to send in thousands of semi-random guesses and collect any email addresses that came back. Some of those addresses were for people with addresses from domains like faa.gov and us.army.mil.

The hackers claim they reported the flaw to AT&T before sending their discovery to the fine folks at Gawker. AT&T, on the other hand, was not pleased to see their security problems appear in a popular tech blog at all, and had this to say in an email to their iPad customers:

On June 7 we learned that unauthorized computer “hackers” maliciously exploited a function designed to make your iPad log-in process faster by pre-populating an AT&T authentication page with the email address you used to register your iPad for 3G service.

…

The hackers deliberately went to great efforts with a random program to extract possible ICC-IDs and capture customer email addresses.

So who’s to blame for a problem like this? Is it AT&T, or do the hackers themselves deserve some of the blame for the public way they handled their disclosure? Give us your thoughts below.

Almost like clockwork, Apple has consistently announced iPhone updates at their WWDC conference every June.  This year was no exception with Steve Jobs himself presenting the much awaited iPhone 4. Just as in the past, this year’s update introduces a number of interesting and must-have features for mobile developers and aficionados everywhere. Here are some of our favorites:

New Display – a Treat for Sore Eyes
Before today, I could not have told you the maximum amount of detail my retina could process in a given inch of space.  Now, thanks to Steve Jobs, I know: 300 dots per inch. That’s important because the iPhone 4 features an astonishing 960×640 display that resolves to 326 pixels per inch. That’s apparently more than what your retina can physically see, a fact that Apple took to heart when they decided to name this feature “Retina Display“.

More incredible is how this compares to the iPad. The iPad is a significantly larger device, but with a resolution of 1024×768 it only has 28% more pixels.

Of course, this also means that Apple developers now have three resolutions to worry about when they’re developing apps: iPhone 4, iPad, and iPhone/iPod 2G/3G/3GS.

Read more…

If you live in the United States and own and iPhone, chances are you’re aware that AT&T has some reliability issues.  As the sole mobile service provider for iPhones in the US, AT&T has seen their usage skyrocket and their reliability crumble in the past few years. Everyone has an opinion about why, but as of today AT&T has revealed what they see as one big problem: heavy bandwidth usage.

Starting this month, AT&T will now limit the amount of data an iPhone or iPad may consume in a given month.  The caps start at 200MB, but for an additional fee you can have all the way up to 2GB.  Existing users will continue to have an unlimited data plan for now, but new contracts will not have that option going forward.

AT&T says that 98% of their customers use less than 2GB of data per month, which means that most people won’t hit the top usage cap.  However, many people will still opt for the lower 200MB data cap because it will actually be cheaper than their current unlimited iPhone data plan.  That means software developers have an interesting new challenge on their hands – bandwidth optimization.

Read more…

As the smartphone battle heats up, a debate has begun around a seemingly crucial question: which platform is faster?  In a lot of ways, that’s impossible to answer. Performance comparisons depend on many factors, including the tradeoff between performance and battery life.  But that hasn’t stopped some from having the debate anyway, and the battle lines right now are over Android’s JVM vs. the iPhone’s Objective C objc_msgSend().  Let me explain.

Android is a Java based platform and uses a Java Virtual Machine or JVM to execute apps (Android’s JVM is called Dalvik). Historically, Java was considered to be a slow and cumbersome platform.  The joke was write once, run anywhere very, very slowly.

So people are saying that Android is slow, right?  Actually, no.  In fact, Java has been optimized so extensively in the past 10 years that its performance is now incredibly fast.  Dalvik has been optimized even further for mobile devices, so Android is one fast platform. What people are actually saying is that the iPhone is slow.

Read more…

Worldwide productivity surely took a nose-dive today, as thousands of worker bees (like me) discovered that Google was featuring the classic Pac-Man arcade game on its ever-changing homepage. What began as a scholarly search for “regression testing tips” quickly devolved into “five” minutes of ghost-chomping fun – but don’t tell my boss.

In honor of Pac-Man’s 30th birthday, Google developed the application (in what we presume is HTML5) to look, sound and behave just like the original version from 1980. [UPDATE: Here's how to download the Pac-Man game for free] We’re not yet sure if this includes the infamous Pac-Man kill-screen bug, but I am determined to find out. I’ll work nights and weekends if that’s what it takes. That’s just the kind of dedicated employee I am.

Anyway, since we’re a software testing company, many of us in the office were curious to see how the application would perform on the various mobile devices we have in-house. Here’s a quick run-down of our findings for each device, including whether or not it worked, along with a few notes:

Read more…

Apple’s recent changes to their developer agreement have unleashed a torrent of anger, hate, and divisiveness on the Internet (which, to my knowledge, has never happened before).  To summarize, Apple announced that the only languages that can be used to develop applications for the iPhone are Javascript, C, C++, and Objective C.  This change was seen as a slap in the face to Adobe who was developing a Flash-to-iPhone app converter that would have made it easy to migrate a Flash application to the iPhone.

Through all of this bitterness, many have argued that Flash is ready for the deadpool – some even cheering its demise.  I disagree.  Actually, I believe just the opposite is true.  Here are 5 reasons why Flash won’t be going away anytime soon.

1. HTML5 is still very immature.
HTML5 is everyone’s favorite choice as a Flash replacement. Read the comments sections on just about any blog or article about this topic, and HTML5 is often hailed as the greatest thing to happen to computing since Apple “invented” the mouse (with Xerox’s help).  The problem with HTML5 is that it’s still an immature and unfinished platform.  While it’s supported by the very latest versions of Firefox, Safari, and Chrome, it’s not yet fully supported in Internet Explorer (although IE9 will bring support eventually). If most of the browsers on the web don’t yet support HTML5, it’s not a fully supported standard.

Read more…