Software Testing Blog

When we relaunched our testing platform last year, we chose to build it in Flex. It allowed us to build a nice web UI, but it also meant that iPhone and iPad owners couldn’t connect to our platform directly. An Apple iOS user had to find a regular computer to report bugs when testing an iOS app.

Well we’ve heard their pleas, and we’re happy to announce something that should help: the uTest iOS app. With a native interface for both the iPhone and the iPad, it’s now possible for testers and customers to test on the couch and on the go.

If that’s enough to make you want to download the app right now, then don’t let me stand in your way. Just click (or tap) on that image to the left and go get it. It’s free, you know.

If you’re still wondering what makes our app special, let me tell you about some of the awesome new things that both testers and customers can do.

For Testers

Customers and testers can start testing with the uTest iPhone app.

iPad users have a native interface that makes full use of the iPad layout.

It goes without saying that the uTest iOS app lets you participate in test cycles and that our app makes it easy to submit bugs right from your iPhone or iPad. But what’s really cool is that if you’re testing another iOS app, you can submit screenshots and videos of your bugs directly from the uTest app. You can even use your camera to take pictures for upload – handy if you need a screenshot of a bug on another mobile device.

In addition to all that, you can do all the other things you would expect while testing, like view the bugs submitted by other testers, reply to tester messenger conversations, and even check out your uTest earnings.

Of course, all this assumes you’re already a uTester. Because if you’re not, you can actually signup for a tester account right there in the app. It will even help you setup your iPhone or iPad as your first testing device on the uTest platform.

For Customers
If you’re a uTest customer, you’re going to love the uTest app. Why approve and reject bugs from a boring old computer when you can do it from the beach? And if you don’t have a beach nearby, how about the comfort of your couch? In fact, you can now review your test cycles from anywhere you like (assuming there’s a phone or wireless signal, of course).

With the uTest app, you’ll also be able to review attachments and even ask testers questions with tester messenger. Everything you need to keep an eye on a test cycle is available at your fingertips.

Wrapup
Of course, our community rigorously tested our iOS app and they discovered over 60 bugs before launch. Their diligence made this app super solid, and that helped us to get approved by Apple for the App Store in record time without having to resubmit.

Now that we’ve launched our first iOS app, we’re hardly finished. We want your feedback and ideas about how we can make it even better. uTest community members can join our tester forums and check out our Platform Feedback section. Customers can contact their project manager directly or drop us a line.

What a year it has been for the uTest platform. In April, we relaunched our product with a ground-up rewrite. Then in the following months we added features like tester snapshots, better reporting, public tester profiles, and more. It’s been a whirlwind of activity to say the least.

Social Sharing
With version 3.3, we’re introducing a new feature to make it easy for testers to share what’s happening with their uTest accounts on Facebook, Twitter, and LinkedIn. For example, you can let the world know that you’ve just gotten a rating badge or that you’ve published your public profile. You can even add a personal message to anything you share.

Remember: You cannot say anything about specific customers outside of the uTest platform, so please be careful about what you write.

Bug Fixes
With so much development in the past 8 months, it was inevitable that some glitches got through. Fortunately, we have a community of 30,000+ testers on the watch, and their efforts gave us a good list of things that needed fixing. The holiday months of 2010 proved to be a great time for bug fixing and product tightening, and with this release we’ve fixed almost 100 bugs – nearly all of them discovered and reported by the talented testers in our community.

Closing Thoughts
Our engineers have been laying the foundations for some really amazing new features that are due to come out in the next few months. It’s been a great year, and we’re looking forward to an even more exciting 2011.

Have an idea for future product releases? uTest community members can join our tester forums and check out our Platform Feedback section. Customers can contact their project manager directly or drop us a line.

Autumn has come and gone in New England. And now that we’re done gawking at the trees, we can get back to the business of building a software testing company. Last night we launched version 3.2 of the uTest platform, and as releases go this one is pretty awesome. We’ve got some fantastic new features to tell you about:

Decline a Test Cycle
Sometimes our testers get invitations for test cycles in which they just can’t participate. Maybe it’s because they’re busy or perhaps they don’t think they’re a good match for the project. In the past, testers had no way to indicate they would not be joining a test cycle – making it hard for us to make sure enough testers were committed to a particular project to get the necessary testing coverage for our customers.

That’s why we’re now enabling testers to decline a test cycle. The process is simple. If you’re invited to a test cycle you won’t be able to join, just decline it. You can even click on a “decline” link right in the invitation email. By doing so, you let the uTest project managers know that they need to invite other testers to fill out a test cycle.

The opposite of declining a test cycle is accepting a test cycle, something testers have been able to do for a while. Remember, if you accept a test cycle, we fully expect you to participate in-scope and in the stated test cycle duration. If you aren’t sure whether or not you’ll be able to participate, we want testers to decline it so we can plan accordingly.

Add a uTest badge to your website by logging in to the uTest platform and choosing from 6 different designs.

Show off your uTest cred with your own public profile.

The new uTest platform homepage.

Tester Public Profile & Badges
For a long time, testers have wanted a way to show off their membership and rating in the uTest community (everyone likes to brag, right?). But in the past, sharing their status with the world has been difficult. Now you can show off your uTest street cred easily with a public profile. This means our beloved testers can even add a uTest badge to their personal website, blog, resume or email signature.

This feature is opt-in. That means you must turn it on to use it (we wouldn’t share your personal information without your permission.) But our testers are awesome, and we want you to show the world that you’re a uTester and all you’ve accomplished with us. To create your public profile, just log in to the uTest platform and click on your Account Settings. Choose the Public Profile tab and configure your profile however you like.

New Platform Homepage
Next time you log in to the uTest platform, you’ll also get to see a new homepage. We want to make it easy for you to keep up-to-date with all the things happening within uTest that matter to you. With this new homepage, you can see your test cycles, bugs, alerts, and messages in one convenient place. Testers will be able to see how they rank within the community, and customers will have visibility into how their test cycles have performed over time.

Office 2007 and RAR Attachments
One last thing we want to announce is that we’ve added support for attaching Microsoft Office 2007 and RAR files to bugs and test cycles, including .docx, .xlsx, and .pptx files. This was a frequently requested change by both our testers and our customers, and we’re happy to oblige.

Have an idea for future product releases? uTest community members can join our tester forums and check out our Platform Feedback section. Customers can contact their project manager directly or drop us a line.

Didn’t we just introduce something new, like last week? Don’t our engineers sleep? Surely we’re not already rolling out more new stuff for our testers and customers?

Well it appears we do have an update, and this one is pretty sweet. If you remember, we launched a whole new uTest experience back in April with version 3.0 of our testing platform. Since then, our product team has been watching and listening to both our testers and customers. We cheer when things work well, but we also take good notes when we see our users struggle.

Today’s update is all about fixing some of those nagging usability issues and making our user experience smoother. Some of the changes are big and some are small, but all of them were inspired by you! Let’s take a look at a few of the things that have changed:

More Effective Testing Coverage
One thing we’ve heard about creating a test cycle is that specifying testing coverage should be easier. The problem is we have so much diversity in our community – with close to 30,000 testers, hundreds of thousands of devices, dozens of operating systems, and several hundred countries and geographic areas. Customers had to do some digging to make sense of all this choice and get exactly the coverage they needed.

Today we’ve made some big improvements in this area. The first change is that you’ll be able to search for countries by global region, so if you’re just interested in European countries, you won’t have to find them in a huge list of every country in the world.

The second change is that we’ve also added quick search boxes that will help you narrow down any field to exactly the coverage choices you need. Whether you want to test your app with the Motorola Droid in the United States on Verizon, or on the Apple iPhone 4 in France on Orange Telecom, you’ll be able to get your test cycle started in a snap.

New Test Cycle Creation Wizard
But it’s not just coverage selection that’s changed. We’ve redesigned our test cycle creation process, simplifying some steps and improving the layout in others. We think that creating a good test cycle is the most important thing a customer can do to ensure testing success, and this update should make this process much smoother.

This new process also means changes for testers who are viewing the test cycle. For example, customers and testers will both appreciate the new Scope and Instructions page that is shorter, simpler, and cleaner.

All the Rest
There are dozens of little things that have improved as well. Here are a few of them:

• You can now easily distinguish read and unread bugs and messages. Unread bugs and messages will be highlighted, while read bugs and messages will be grayed.
• If you use a bug tracking system, newly exported bugs will appear much cleaner and easier to read.
• Both testers and customers can now choose to stay signed-in to the platform without having to reenter their password each time.

Do you have an idea for our future product releases? uTest community members can join our tester forums and check out our Platform Feedback section. Customers can contact their project manager directly or drop us a line.

A couple weeks ago, we rolled out v3.0.4 of our platform. And now, after a little extra time in the oven, we’ve launched another feature designed to make it quicker & easier to access your uTest account.

Social Sign-In
Let’s say you’re like millions of other people around the world and you have an account on Facebook, Twitter, or LinkedIn. And let’s also say you’re tired of keeping track of passwords, remembering to sign-in everyday to all the websites you like, and are yearning for something simpler. Starting today, we have a solution for you, at least when it comes to your uTest account (we can’t speak for anyone else). Now you can log in to the uTest platform using the credentials from your Facebook, Twitter or LinkedIn accounts.

Setting up this linkage is simple:

1. Get started by visiting our platform and clicking the button for your favorite social network(s).
2. Then sign in to your social network and confirm the connection.
3. And that’s pretty much it. The next time you visit the uTest platform, just click that same button and you’re in.

Enjoy!

For the past several months, we’ve been knocking off some highly sought-after platform features, and v3.0.4 is no exception. And with this latest release, we’ve made it even easier to choose the perfect testers for your project and integrate with more of the top bug-tracking systems.

Easier Tester Selection

When our customers create a new test cycle, we want the process to be as simple as possible. But we also want to put the highest degree of power and precision in their hands when it comes to building their testing team. One of the more common feature requests we’ve heard is to make it even easier to select testers for private test cycles. Our product crew has heard your cries, and we’re pleased to introduce a dramatic overhaul of the tester selection process when creating a test cycle.

We realized that every customer has a different set of needs when it came to finding exactly the right set of testers for their product. So we made it easy to search within testers and handpick the best candidates. After a test cycle has launched, the tester selection will even tell you which invited testers have accepted your testing invitation. Need more testers in a certain region? Finding and adding more will be a snap. Check it out!

More Bug Tracker Integration
More and more companies are using bug tracking systems, and we’ve long made it easy to move bugs from uTest into several popular systems. Until recently, we’ve supported Bugzilla, Jira, and Rally. With this release, we’re adding support for Mantis. We’ve also added support for Jira Studio, and we’ve improved our support for Rally. Lastly, our engineers have even built integration with the bug tracking systems of some of our biggest customers.

Is your favorite bug tracking system not yet supported? Let us know, and we’ll add it to our list.

More Handsets for Tester Profiles
If there’s one thing that’s true in this world, it’s that gadget makers never stop making new gadgets, and that gadget lovers never stop buying them. We’ve added a whole bunch of new devices to our database, including the iPhone 4, the iPad, and more. If you are a tester who has the latest and greatest mobile phone or tablet, make sure you update your tester profile with that information so we can assign you projects.

All the Rest
That’s not all! We’ve also worked on a couple of other smaller, but still important features:

• Search for test cycles by their ID
• Misspelled words are now highlighted in text boxes
• Dozens of bug fixes and behind-the-scenes improvements

Have a great idea for our future product releases? uTest community members can join our tester forums and check out our Platform Feedback section. Customers can contact their project manager directly or drop us a line.

New Englanders aren’t known for doing well in hot weather, and a recent heat wave has kept us all inside in the air conditioning. Since our engineers can’t go outside without melting or vaporizing, they’ve been inside keeping busy creating some cool new features. Let’s take a look:

Attachment Viewer
Our testers love uploading attachments with their bug reports.  Whether it’s a screenshot or a video, testers know that an attachment can make a bug report easier for our customers to understand and visualize.

Until now, customers have had to download every attachment individually and then open them on their desktops. To make this process faster and easier, we have launched a nifty new attachment viewer that works inside our platform. Now customers and testers can view video and image attachments with a single click, toggle between them easily, and download the ones you want to keep. And for non-multimedia files like spreadsheets and documents, we still make it easy to download the files and open them from your desktop.

Take a look at this short 30 second video showing off the new attachment viewer:

Read more…

Late last night, our engineering crew released version 3.0.2 of the uTest platform. This update contains several larger features and a bunch of small ones — all based on insights we gathered from our customers and testers. Here are a few of the highlights:

Favorite Tester
As customers run more and more test cycles, they often identify favorite testers they want to invite back again and again. These recurring relationships are good for customers and the testers alike. In the past, customers had to keep track of their favored testers by hand. But with our new favorite tester feature we’re making it easy for customers to remember the testers they like best, and to invite them back over and over.

Customers can now more easily see which testers participated in their past test cycles as well as how they performed. By clicking a yellow star icon next to a tester’s name, a customer can mark that particular tester as one of their favorites. When that customer runs a new test cycle, their favorite testers are placed at the top of the list, making inviting them back a snap.

Tester Snapshot
Wouldn’t it be nice for testers to have a place to show off their ratings along with an overview of their location, testing environment, and maybe a photo of themselves? With this launch we’ve made that idea a reality with our new tester snapshot feature.

Testers and customers can click on any tester’s name in the platform and see the public tester snapshot for that tester (location, testing environments and a photo). Testers who are rated or who have been marked as a favorite tester will also have some snazzy new badges that show off their uTest skills. This information can be viewed by other testers or customers.

If a tester clicks on their own name in the platform, they can also see their private tester snapshot (including past performance and earnings history). Just click “Statistics” and your private tester snapshot will display your past test cycles, your bug approval percentages and more. This information cannot be viewed by other testers or customers.

All the Rest
Other improvements include:

• Improved workflow for submitting and approving test cases
• More intelligent tester notification emails
• Numerous performance tweaks
• Dozens of bug fixes and other minor enhancements

Have a great idea for our future product releases? uTest community members can join our tester forums and check out our Platform Feedback section. Customers can contact their project manager directly or drop us a line.

After months of hard work, we’re excited to announce version 3.0 of our testing platform.  This is much more than a simple refresh or a minor upgrade — this is a full-blown rewrite of our testing platform, from the UI design, some cool new features, back through the code and a new, open architecture.  And it’s all based upon feedback from customers, testers and all we’ve learned from 1,000+ test cycles in the past 18 months.

Our goal was to produce a faster, more usable, more powerful experience for uTest customers and testers alike.

So what’s new?

• User Interface: Our new UI was designed to provide a simpler, richer, more interactive experience with more intuitive navigation from bug-to-bug or test-cycle-to-test-cycle
• Improved Infrastructure: Our new architecture enables better scalability and, more importantly to you, is designed to provide improved performance (with faster page load times) around the world
• API and Integration: Our new and open APIs will enable us to rapidly expand our offering and better integrate with our customers, partners and third-party app developers

What’s in it for you?  If you’re a customer, keep reading.  To check out what’s new for testers, you can skip down to the tester features.

Customer Features
Once our customers get past our sleek new UI, we hope they notice the great new features and changes we’ve included.  Two features are really important.

First, we’ve built a new test cycle wizard.  Creating a clear, concise test cycle is the most important thing a customer can do to ensure success, but there are a lot of details required to make a test cycle perfect.  Our new wizard makes this process easy, making sure each part of the test cycle is well-documented.

Second, we’ve greatly enhanced our tester rating system.  Our platform will evaluate the past performance of each tester based upon activity levels (# of test cycles participated in, # of bugs reported, # of test cases completed) and the quality of that activity (bug approval %) and, once we have enough data points, we’ll assign them a rating.  Testers who are in the top 20% of all rated testers will receive a gold, silver, or bronze rating.

Some of the other new features we’ve added in v3.0:

• Resizable Columns – On any table or list, all columns can be re-sized and re-arranged by customers using a simple drag-and-drop.
• Tester Messenger - Enhanced tester messaging tools help you communicate with testers to get what you need from each bug report
• Testing Coverage Reports – Be confident that you’ve covered every corner of your testing matrix with our new coverage reports. You’ll know what’s been tested and what still needs to be tested in a single informative report
• Easier Filtering – All test cycles and bugs can be easily filtered to find exactly the information you need
• Smart CSV Exporting – Export a CSV file with just the information you need about your bugs or your test cycles
• Multiple Users – Add multiple members of your testing organization easily with our new account management tools

Tester Features
Besides the new UI, we’ve added a lot of cool new features for our uTester community.  However two features are really huge for testers.  The first is our new test cycle wizard that helps testers review test cycles more easily and completely, as well as telling uTest whether or not they intend to participate in that test cycle.  Now when a tester checks out a test cycle, they will be able to indicate whether or not they want to join.

Our second big feature is a new & improved tester ratings system.  In the past, testers were rated with a star system based upon subjective customer feedback.  Starting today, testers who are active will earn a rating and a “Rated” badge.  This shows that these testers are actively participating in test cycles contributing their expertise to the uTest marketplace.  Additionally, those testers who are in the top 20% of our rated testers will receive a gold, silver, or bronze badge.  The remaining testers will receive a “Rated” badge.  Testers who are not active or for whom there is not enough data will not receive a badge.  New testers will receive a “New” badge.

A tester’s rating will depend on many factors including total activity and recent activity (# of cycles participated in, # of bugs reported, # of test cases completed), as well as the quality of that participation (bug approval %, test case approval %, and accuracy in specifying bug types and severity).

Other great new features include:

• Resizable Columns – For any table or list, all columns can be re-sized and re-arranged using a simple drag-and-drop.  For example, a tester can move the test cycle name to be in the first column with a width that displays the entire test cycle name
• Tester Messenger – Enhanced communication tools between testers, uTest project management and customers
• Easier Filtering – All test cycles and bugs are easily filtered.  For example, testers may filter bugs by “My Bugs Only,” status, severity, and by test cycle; even for closed test cycles
• Clearer Test Cycles – All test cycles are formatted in a more digestible way to help testers more easily and completely review each test cycle
• Improved Attachments – Bug attachment size limit has been raised from 5MB to 10MB (there is a limit of three attachments per bug).  Also, more file types are permitted for upload.
• Smarter Emails - Cleaner, more informative notification emails from the uTest platform

Wrap-up
Our engineers and product managers have worked countless hours putting together version 3.0, but we also want to thank our customers and testers for their countless ideas and insights.  As we continue to improve and refine our platform, we are always open to thoughts and ideas about how to make our product better.

Have a great idea for our future product releases?  Testers should join our testing forums and check out our Platform Feedback section. Customers can contact their project manager or drop us a line.

A heads-up to our testers and customers — this weekend, we’ll be launching v3.0 of the uTest platform.  This will be a major, major (did I mention major?) upgrade of our product.  It also means that we’ll be taking the platform offline for a period of time beginning on Saturday morning.  We’re working closely with customers to minimize the number of test cycles that will be live during this period, but we do apologize for any inconvenience that may result from this planned downtime.

The changes in this new version will be obvious for all to see.  But we’ll publish a complete write-up of all the changes for customers and testers, including artwork to highlight what’s new & improved.  Stay tuned to the uTest blog or our uTester forums for updates on how the launch is going on Saturday.

UPDATE: As we approach noon EST, things are progressing well.  We’re still running down our long list of launch tasks, but things are looking good at this point.  More details coming soon.