Software Testing Blog

Here at uTest, we have the unique privilege of seeing some of the world’s latest, greatest mobile apps before they make it big. After years of testing these apps, we’ve also gotten pretty good at spotting the likely winners and losers through several distinguishing characteristics. These include functionality, security, ease-of-use and most importantly: originality.

As you’re probably aware, not every app is original. In fact, most of the apps in existence are anything but unique. Scroll through your app store for five minutes and see if you can argue otherwise.

Anyway, this glut of apps – especially copycat apps – has gotten so ridiculous that it’s got several high-profile tech bloggers essentially begging to make it stop. Here’s an excerpt from Can We Stop The Copycat Apps from Rip Empson of TechCrunch:

While I encourage developers to continue making great apps, I do question the need for both making and for approving the parade of — for lack of a better word — “rip-off” apps. What am I talking about? Example: Over the last week, I’ve watched another fairly blatant copy of Angry Birds hover inside the “Top Free iPhone Apps” list on the App Store, even grabbing the second spot at one point.

I’m not naming the app explicitly, because I don’t want to give the game free publicity. That’s what they want, and it’s probably a good idea to avoid promoting the production and downloading of spammy (cr)apps. But needless to say, the scenario is familiar: The game’s icon is practically identical to that of Angry Birds, it has “Angry” in the title, the design and gameplay — while not exactly identical — have enough similarities to make for some serious eye-rolling. Not to mention, the game is awful. One-star reviews abound.

Read more…

Tell me if this sounds familiar:

1. Company X dreams up a new app (or a new version)
2. Product management specs the requirements; engineering builds it, and QA tests it
3. As soon as it launches,  Twitter and Facebook blow up with user complaints about how the app doesn’t work; is confusing to use; is slow; or some other flaw
4. Everyone blames engineering and QA for launching something that sucks

So what went wrong?  We need better test automation… we need to outsource testing… we need to partner our testers with our engineers… we need better product documentation… we need to move from SCRUM to Kanban.  We invest thousand of hours and millions of dollars trying to create apps that work as designed — not just inside our firewall, but in the hands of users.

Problem is, we focus all of these good intentions on stuff that occurs inside the lab (whether it’s our in-house QA lab, our test automation, or a vendor’s test lab 12 timezones away).  And users don’t consume our app under the sterile conditions of a lab environment. So how do companies ensure that their apps work as well under real-world conditions as they do under lab conditions? There are two ways:

1. Running a beta program
2. Push a portion of their testing out into a distributed community

And as MG points out over at TechCrunch in a recent post about Apple’s launch of Siri, the use of the “beta” label no longer lets companies off the hook with regards to app quality:

Read more…

As you may have read on Monday’s blog post, uTest launched a new informational campaign to promote http://www.inthewildtesting.com.   The web site – and associated social media channels, including a Twitter profile – are intended to educate forward-thinking technology leaders about the necessity, benefits and real use cases of in-the-wild testing. 

We decided to launch it at TechCrunch Disrupt in San Francisco because the very concept of in-the-wild software testing (versus traditional methodologies) is, well…disruptive. 

Sure enough, TechCrunch Disrupt turned out to be the perfect event!  There were more than 2,600 innovative, entrepreneurial-minded techies, investors and exhibitors (35% more attendees than expected) filling the halls of the Design Concourse Center from Monday to Wednesday.  In its usual fashion, the conference itself attracted top industry-leaders such as Reid Hoffman of LinkedIn, Marissa Mayer of Google, Vinod Khosla, and even Ashton Kutcher.

uTest hosted a ton of terrific activities over the course of the event:

Read more…

It turns out that launching a service as monumental as uTest Express from the show floor at CTIA Wireless is a good move! It garnered incredible national and international media attention — from TechCrunch (and TC EU), ReadWriteWeb (and RWW Spain) and The Marker Israel to leading mobile publications WirelessWeek and RCRWireless to Boston’s top local blog BostInnovation.

Here’s a peek at the RCR interview with our CMO, Matt Johnston:

Of course we weren’t the only ones launching new products and services from sunny Orlando, FL.

Let’s start with new smartphones (yes, even waterproof smartphones!). Much like CES and Mobile World Congress, CTIA was fertile ground for the release of cool 3D phones such as the LG Thrill 4G and the HTC Evo 3D, and T-Mobile’s G2X which was also named CNET’s Best Phone.

And we can’t forget the new tablets. Samsung released the “world’s thinnest tablets”! The Samsung Galaxy 8.9 and 10.1 tablets are 8.6 millimeters thick, thinner than the iPad 2 (8.8 millimeters), earning them CNET’s Best Tablet. Check out the other 8 tablets revealed at CTIA and reviewed by PC World here.

If you weren’t able to make it this year, here are some pics from the show floor. I was personally moved by NTT Docomo Japan’s empty 40×40 space this year. As FierceWireless put it, “a bare floor decorated with a simple vase of the white, cherry blossom flowers with a call for funds to support the victims, was a pitch-perfect show of support and good taste.” (Pics after the bump.)

Read more…

Last week, a familiar theme re-took center stage.

Like Magic vs. Bird; like Tupac vs. Biggie; and like the Brooklyn Dodgers vs. the LA Dodgers before them.  TechCrunch contributor and leading academic, Vivek Wadhwa, and Boston Globe journalist, Scott Kirsner, squared off in a classic east/west battle that rippled across the Twittersphere and bloggerverse.

A bit of personal background before I share my thoughts:

1. I was not in attendance at last week’s MIT event, so my take is based upon their subsequent posts and tweets
2. I’m an avid reader of both Kirsner and Wadhwa, and have heard both present in person numerous times
3. I spent eight years in NYC’s startup scene; lived in Boston for the past four years; and I’ve spent about 1/4 of my time in the Valley for the last five years (I also grew up in the tech hot spot of Kalkaska, Michigan if that helps)

But after several days of watching this debate bubble up and boil over, here’s my take from the front lines:

Meh.

Yeah, that’s it. I’ve listened to these arguments from Wadhwa before and I find them mildly interesting and, at times, true. I see Kirsner taking on the mantle of defending the Boston startup scene and a small part of me cheers his call for meaningful proof points.

Conversely, when I hear Boston-area entrepreneurs being defended as equal to their NYC or Valley counterparts, I’m mortified to discover that some believed I needed to be defended in the first place. But more than anything else, I think the debate misses for 5 reasons:
Read more…

With all the attention and praise that mobile apps have received over the last few years, it’s easy to forget about the progress of their web and desktop cousins. Thanks to a recent guest post on TechCrunch, the software community received a terrific reminder that App Store innovation will NOT be confined to the world of mobile.

Written by Evernote CEO Phil Libin, Four Lessons From Evernote’s First Week On The Mac App Store offers some keen insight into the evolving world of app stores, including the impact it’s having on development efforts, user preferences and more. It’s well worth a read, but here are a few excerpts to get you started (key takeaways in bold, emphasis mine).

Over the past year, about 70% of Evernote’s new users came from mobile app stores, mostly iOS and Android. This led us to the understandable conclusion that mobile was the crucial thing that made a platform attractive to independent developers. Last week made us realize that the reality is a little bit more nuanced. It isn’t mobile that’s overwhelmingly important, it’s the app store. Until a week ago, all the good app stores just happened to be on mobile devices, but someone with a shiny new Macbook is just as eager to get the best apps as someone with a shiny new iPhone.

It’s obvious in hindsight, but the presence of a well-formed app store is the single most important factor for the viability of a platform for third party developers. If you want to take this a step further and say that a robust third-party software market is the most important factor for the success of the platform overall, well…

The immediate popularity of the Mac App Store, and the iPhone App Store before it, reinforces my belief that in a world of infinite software choice, people gravitate towards the products with the best overall user experience. It’s very hard for something developed in a cross-platform, lowest-common-denominator technology to provide as nice an experience as a similar native app.

Read more…

As you probably heard, uTest closed a $13MM C Round last week. We were pretty stoked and, in our post-celebration bliss, we put out an open call for questions about the process of fundraising. Well, our friends, readers and followers did not disappoint. Check out the below list of questions and the off-the-cuff responses from our CEO and co-founder, Doron Reuveni (@doronr).

Is this the first company you’ve started, and what led you to launch your own startup?

DR: My co-founder and I really came from two different spaces. He came from more of a testing and QA background, while I came from more of a business and engineering background. But we both felt the same pain: We would spend so much time and money on improving the quality of software with the standard processes and tools, yet there would still be lots issues found by the customer. Eventually, we asked ourselves, “What if we could find a way to test software under live conditions, on-demand?” That’s really how uTest came about. And yes, uTest is the first company I started.

What was the a-ha moment that triggered the idea behind uTest?

DR: My co-founder and I had both experienced the pain, cost and frustration of trying to test software via traditional means. We were doing everything right – in-house, outsourcing, automation, documentation – but unexpected bugs and defects still emerged when we put it in the hands of users.  Every single time.

So we started thinking about moving testing outside the lab and into the wild.  And the only way to do that was through a crowdsourced model. That really got everything started.

What have been three major obstacles that you had to overcome as an entrepreneur?

DR: I would say the number one challenge – because I don’t consider them to be obstacles – was to sell the original idea to potential investors. At the time we started looking for our first round of funding, all I had was a PowerPoint presentation and a smile. For that first round, we didn’t really have any prototypes – and we certainly didn’t have any revenue – so it was very much a challenge to talk with some of these potential VCs about the theory of crowdsourcing and how it fit the software testing space.

I would say the second biggest challenge has been recruitment, which is of course an ongoing challenge. At the end of the day, it’s really the people that make the difference – not the product or the technology or the marketing – so we made it a priority to recruit and retain high-level talent and to develop a company “culture.” These things are a lot more difficult than they sound, especially when you are expanding internationally.

The third major challenge we had to overcome was managing revenue and customers with a maturing platform. We made it a goal to get the platform to market quickly, and so we already had a few beta customers by the time we launched. We didn’t really have time to refine a lot if it before our customers got to use it. We were getting tremendous amount of feedback almost instantly, but this puts a lot of pressure on you to improve the product at a pace you can’t really keep up with at first.

Once you starting add customers and revenue to the equation – especially for a technology startup – it completely changes the dynamic.

Read more…

As you may have heard, we raised a $13 Million C Round led by ScaleVP which we announced yesterday and couldn’t be more excited about the things to come!

The media definitely re-affirmed and fueled our enthusiasm because leading journalists, bloggers, and social media mavens (500+ tweets!) were buzzing with the news. If you’d like to catch up on the details, here are our top biggest press hits:

1. TechCrunch by @leenarao
2. Xconomy by @wroush
3. Huffington Post/Silicon Alley by @courtneybmyers
4. Dow Jones Venture Wire by @tomiogeron
5. Boston Business Journal/MHT by @galenmoore
6. FINS/The Wall Street Journal by John Shinal

For all news, please visit the uTest News page. So, if you get a quick coffee break today, please join us and “Read All About It!” and let us know if you have any questions about our upcoming plans and expansion. Thanks again to all for your notes of congrats!

*Disclaimer: This post is a bit of a shameless self plug for all our moms out there.

The Twitter bird has been seen a lot more than the Fail Whale in the past few month — a testament to the company’s investments in infrastructure. But now, a third species has jumped into the Twitter spotlight: the bug.

And while this particular bug has since been squashed by Twitter’s engineers, it’s still an interesting defect. As all of the major new media heavyweights noted, this Twitter bug briefly enabled the brevity-challenged among you to stretch your legs and break free from the shackles of 140 characters. To put it more plainly, let’s bring in Alexia Tsotsis (@alexia) from TechCrunch:

The Twitter bug which has left many befuddled is exploiting a length limit flaw in the new t.co URL shortener, allowing users to tweet out non-URL links of outrageously more than 140 characters

If you’d like to reproduce the effect, and it seems to be catching, you can visit http://twitter.com/share?text=&url=yourtext, add whatever you want in place of “yourtext,” copy and paste your new t.co URL to Twitter and long tweet away.

Update: Looks like the nimble engineers at Twitter have disabled the feature within the hour this post went up, much to everyone’s dismay. Scripting News’ Dave Winer went so far as to create a web app for the Fat Tweets.

For those of you who follow Twitter (sorry, couldn’t resist), check out the entire TechCrunch article, as well as Mashable’s take.

So maybe it wasn’t AT&T’s fault after all.

Apple recently revealed that there is a fundamental flaw in their method for calculating how many signal bars to display.  And we have the iPhone 4 (and its “learn to hold your phone the right way” fiasco) to thank for bringing this software snafu to light.

CNN Money shares the following details from Apple:

“Upon investigation, we were stunned to find that the formula we use to calculate how many bars of signal strength to display is totally wrong,” Apple wrote in a statement posted on its website. “Our formula, in many instances, mistakenly displays 2 more bars than it should for a given signal strength.”

That means, for example, that iPhones sometimes display four bars when they should be displaying two. Apple said users reporting a significant drop in bars when they hold their iPhone 4 are probably in an area of “very weak signal strength” but were unaware of that because the phone displayed four to five bars.

“Their big drop in bars is because their high bars were never real in the first place,” the company said.

Perhaps most surprising, Apple disclosed that the problem is not confined to the iPhone 4.  The faulty formula has been present in every iPhone model since the 2007 original.  Questions remain about whether the issue is strictly software-related, or if it also involved hardware problems.  However, Apple has said it will release a free software update in the next several weeks to fix the glitch. It will use a new formula recommended by AT&T.