Software Testing Blog

Late last night, our engineering crew released version 3.0.2 of the uTest platform. This update contains several larger features and a bunch of small ones — all based on insights we gathered from our customers and testers. Here are a few of the highlights:

Favorite Tester
As customers run more and more test cycles, they often identify favorite testers they want to invite back again and again. These recurring relationships are good for customers and the testers alike. In the past, customers had to keep track of their favored testers by hand. But with our new favorite tester feature we’re making it easy for customers to remember the testers they like best, and to invite them back over and over.

Customers can now more easily see which testers participated in their past test cycles as well as how they performed. By clicking a yellow star icon next to a tester’s name, a customer can mark that particular tester as one of their favorites. When that customer runs a new test cycle, their favorite testers are placed at the top of the list, making inviting them back a snap.

Tester Snapshot
Wouldn’t it be nice for testers to have a place to show off their ratings along with an overview of their location, testing environment, and maybe a photo of themselves? With this launch we’ve made that idea a reality with our new tester snapshot feature.

Testers and customers can click on any tester’s name in the platform and see the public tester snapshot for that tester (location, testing environments and a photo). Testers who are rated or who have been marked as a favorite tester will also have some snazzy new badges that show off their uTest skills. This information can be viewed by other testers or customers.

If a tester clicks on their own name in the platform, they can also see their private tester snapshot (including past performance and earnings history). Just click “Statistics” and your private tester snapshot will display your past test cycles, your bug approval percentages and more. This information cannot be viewed by other testers or customers.

All the Rest
Other improvements include:

• Improved workflow for submitting and approving test cases
• More intelligent tester notification emails
• Numerous performance tweaks
• Dozens of bug fixes and other minor enhancements

Have a great idea for our future product releases? uTest community members can join our tester forums and check out our Platform Feedback section. Customers can contact their project manager directly or drop us a line.

After months of hard work, we’re excited to announce version 3.0 of our testing platform.  This is much more than a simple refresh or a minor upgrade — this is a full-blown rewrite of our testing platform, from the UI design, some cool new features, back through the code and a new, open architecture.  And it’s all based upon feedback from customers, testers and all we’ve learned from 1,000+ test cycles in the past 18 months.

Our goal was to produce a faster, more usable, more powerful experience for uTest customers and testers alike.

So what’s new?

• User Interface: Our new UI was designed to provide a simpler, richer, more interactive experience with more intuitive navigation from bug-to-bug or test-cycle-to-test-cycle
• Improved Infrastructure: Our new architecture enables better scalability and, more importantly to you, is designed to provide improved performance (with faster page load times) around the world
• API and Integration: Our new and open APIs will enable us to rapidly expand our offering and better integrate with our customers, partners and third-party app developers

What’s in it for you?  If you’re a customer, keep reading.  To check out what’s new for testers, you can skip down to the tester features.

Customer Features
Once our customers get past our sleek new UI, we hope they notice the great new features and changes we’ve included.  Two features are really important.

First, we’ve built a new test cycle wizard.  Creating a clear, concise test cycle is the most important thing a customer can do to ensure success, but there are a lot of details required to make a test cycle perfect.  Our new wizard makes this process easy, making sure each part of the test cycle is well-documented.

Second, we’ve greatly enhanced our tester rating system.  Our platform will evaluate the past performance of each tester based upon activity levels (# of test cycles participated in, # of bugs reported, # of test cases completed) and the quality of that activity (bug approval %) and, once we have enough data points, we’ll assign them a rating.  Testers who are in the top 20% of all rated testers will receive a gold, silver, or bronze rating.

Some of the other new features we’ve added in v3.0:

• Resizable Columns – On any table or list, all columns can be re-sized and re-arranged by customers using a simple drag-and-drop.
• Tester Messenger - Enhanced tester messaging tools help you communicate with testers to get what you need from each bug report
• Testing Coverage Reports – Be confident that you’ve covered every corner of your testing matrix with our new coverage reports. You’ll know what’s been tested and what still needs to be tested in a single informative report
• Easier Filtering – All test cycles and bugs can be easily filtered to find exactly the information you need
• Smart CSV Exporting – Export a CSV file with just the information you need about your bugs or your test cycles
• Multiple Users – Add multiple members of your testing organization easily with our new account management tools

Tester Features
Besides the new UI, we’ve added a lot of cool new features for our uTester community.  However two features are really huge for testers.  The first is our new test cycle wizard that helps testers review test cycles more easily and completely, as well as telling uTest whether or not they intend to participate in that test cycle.  Now when a tester checks out a test cycle, they will be able to indicate whether or not they want to join.

Our second big feature is a new & improved tester ratings system.  In the past, testers were rated with a star system based upon subjective customer feedback.  Starting today, testers who are active will earn a rating and a “Rated” badge.  This shows that these testers are actively participating in test cycles contributing their expertise to the uTest marketplace.  Additionally, those testers who are in the top 20% of our rated testers will receive a gold, silver, or bronze badge.  The remaining testers will receive a “Rated” badge.  Testers who are not active or for whom there is not enough data will not receive a badge.  New testers will receive a “New” badge.

A tester’s rating will depend on many factors including total activity and recent activity (# of cycles participated in, # of bugs reported, # of test cases completed), as well as the quality of that participation (bug approval %, test case approval %, and accuracy in specifying bug types and severity).

Other great new features include:

• Resizable Columns – For any table or list, all columns can be re-sized and re-arranged using a simple drag-and-drop.  For example, a tester can move the test cycle name to be in the first column with a width that displays the entire test cycle name
• Tester Messenger – Enhanced communication tools between testers, uTest project management and customers
• Easier Filtering – All test cycles and bugs are easily filtered.  For example, testers may filter bugs by “My Bugs Only,” status, severity, and by test cycle; even for closed test cycles
• Clearer Test Cycles – All test cycles are formatted in a more digestible way to help testers more easily and completely review each test cycle
• Improved Attachments – Bug attachment size limit has been raised from 5MB to 10MB (there is a limit of three attachments per bug).  Also, more file types are permitted for upload.
• Smarter Emails - Cleaner, more informative notification emails from the uTest platform

Wrap-up
Our engineers and product managers have worked countless hours putting together version 3.0, but we also want to thank our customers and testers for their countless ideas and insights.  As we continue to improve and refine our platform, we are always open to thoughts and ideas about how to make our product better.

Have a great idea for our future product releases?  Testers should join our testing forums and check out our Platform Feedback section. Customers can contact their project manager or drop us a line.

A heads-up to our testers and customers — this weekend, we’ll be launching v3.0 of the uTest platform.  This will be a major, major (did I mention major?) upgrade of our product.  It also means that we’ll be taking the platform offline for a period of time beginning on Saturday morning.  We’re working closely with customers to minimize the number of test cycles that will be live during this period, but we do apologize for any inconvenience that may result from this planned downtime.

The changes in this new version will be obvious for all to see.  But we’ll publish a complete write-up of all the changes for customers and testers, including artwork to highlight what’s new & improved.  Stay tuned to the uTest blog or our uTester forums for updates on how the launch is going on Saturday.

UPDATE: As we approach noon EST, things are progressing well.  We’re still running down our long list of launch tasks, but things are looking good at this point.  More details coming soon.

Time flies here at uTest, and not long after we rolled out version 2.8 of our testing platform, our engineering team comes along and delivers version 2.9.  And with this being the holidays, they’ve included some great new features sure to brighten the eyes of software companies and testers everywhere.  So grab some milk and cookies and let’s unwrap these great new version 2.9 features:

Better Test Team Building
As customers come back to test again and again, they have been asking us for greater control in building their recurring test teams.  Starting with version 2.9, customers can more easily find the right testers for their projects, including inviting back the testers they liked best from past test cycles.

That means a customer can get the team they want faster and more easily than ever before.  And that means it’s faster and easier to run new test cycles. 

Cloning a Test Cycle
We’re also introducing the ability to clone a test cycle.  Cloning a test cycle lets customers take all the settings from a previous project – including their testing requirements across locations, environments and testers – and reuse them on a new test cycle with just a few clicks.  That means our customers can focus on only the things they want to do differently – such as adjusting the testing scope or special instructions.

To get started, open the test cycle you want to clone and look for the “Clone Release” button.  Click it, and then change the name, end date, and any other details you wish to adjust in your new test cycle.

Incorporating Test Cases
More and more often, customers are asking us for help, not just in finding new bugs, but also to help make sure the old ones stay gone.  Many customers are now asking our testers to provide testing coverage (across functionality, across locations, across OS and browsers) to validate their applications.  The best way to achieve this testing coverage is by using test cases that ensure that all the important details are covered.  Until now, we’ve asked for testers to submit their completed test cases as “technical bugs”, but from this point forward, testers can submit a completed test case as its own kind bug type.

When submitting a completed test case, report it as a bug and choose “Test Case” as the bug type:

Have a great idea for our future releases?  uTesters can join our testing forums and check out our Platform Feedback section.

Thursday, in the quiet darkness of a cold New England night, the uTest engineering team flipped a switch and lit up computer screens worldwide with version 2.8 of our testing platform.  Featuring a shiny new look that matches our website, blog and forums, this is a big step forward for our platform and our users.

But that’s just the tip of the iceberg!  Version 2.8 also introduces a few highly useful new features that our customers and testers have been asking for:

Real-Time Coverage Reports
We often provide our customers with a testing coverage report after a test cycle has concluded.  With a complete record of what platforms, operating systems, devices, and browsers were tested, the coverage report has provided customers with peace of mind that all the bases were covered.

Starting today, that information is now available in real-time while the test cycle is still running.  This gives testing managers a whole new way to find out what’s been covered and what needs to be covered before the test cycle completes.  Testers can then be redirected to cover important elements that haven’t been sufficiently verified. (Click for a larger screenshot.)

Easier Environment Reporting for Testers
Along with real-time coverage reports, we are also introducing a better way for testers to include their device details (OS, browser, etc.) with each reported bug.  As part of our tester re-profiling from earlier this year, we have a very complete description of each tester’s platform, including all relevant software and hardware.

Now when a tester reports a new bug, they will be presented with a list of the testing environment information from their platform profile.  Instead of describing their environment from scratch in each bug, testers will be able to choose which elements of their environment were relevant for a particular defect.  Bug reporting will be quicker, simpler, and more reliable, while the resulting bug data will be more structured and easier to analyze.

For example, if you are a tester who has profiled that you have a Mac running Mac OS 10.6, you will only see the browsers, firewall and anti-virus tools that you’ve included in your profile (so make sure your tester profile is 100% up to date and accurate).  (Click for a larger screenshot.)

Other Fixes
As always, we have our tester community to thank for their help in finding and fixing a number of bugs.  Have a great idea for our future product releases?  Join our testing forums and check out our Platform Feedback section.

Last night, our development team unleashed version 2.7 of the uTest platform.  This release includes some great enhancements – many of which were based on requests from our customers.  We’re especially excited about a few features in particular:

Enhanced Mobile Releases
Mobile app testing has always been something we’re proud of, but as the number of mobile testers and customers has grown, we’ve outgrown our normal release structure.  With so many mobile phones, carriers, and operating systems around the world, mobile customers have asked us to make it even easier to choose between them for new releases.

Now customers will experience an improved workflow for mobile releases.  We help you select just the right specifications for your testing – meaning you get just the right testers you need for your project.  So whether you’re looking for Blackberry testers in Britain or Android testers in Argentina, we make it easy to get started testing.

(click thumbnail for a larger view)

Niftier NDA Support
While we have long supported non-disclosure agreements between testers and customers, the process has been more cumbersome than we would have liked.  Starting today, customers can upload a non-disclosure agreement to which testers must read and agree before starting a release.  This will greatly simplify the process of using NDAs on releases and make it possible for customers with IP sensitivities to get going even faster.

Customer Notes
Many of our customers have been asking for a way to add their own notations to a bug report separate from the tester’s feedback.  Sometimes testing managers want to share information back and forth within their team, track the progress for bug verification, or capture quick thinking on bug followup.  Customer Notes are private and only visible to Testing Managers and uTest Project Managers.  Individual testers will never see them.

Customer notes are located on the right column of individual bug reports.

Other Fixes and Enhancements
We also made a number of other improvements including beefing up features like .CSV exporting, streamlining some of our back-end project management, and tuning up our platform’s performance to handle larger and larger traffic loads.

Our tester community, as always, offered some great ideas and helped us find and fix a number of bugs.  Have a great idea for our future product releases?  Join our testing forums and check out our Platform Feedback section.

The uTest dev team has been busy (and sleep-deprived) of late.  Last night, we released version 2.6 of our platform, which contains our new and vastly improved tester profile. This will enable us to more precisely match our customers’ testing projects with the right members of the global community.

And for testers, it means we can do a much better job of inviting you to the the releases that match your location, languages, hardware, software, preferences and experience.  It will also translate into fewer emails hitting your inbox and more projects that are relevant to you. Since this revamped tester profile is such a big leap forward, we’re requiring all of our testers to complete it before they can participate in future testing projects.  It’s a simple 4-step process and takes just a few minutes!

If you’re already a uTester, please re-profile here. Not yet a part of the world’s largest community of software testers? You can sign up in minutes right here. Want to see the goods? Here’s what the new tester profile looks like:

Step 1 of 4:  Getting Started
We start by capturing the basic info from new testers (current testers will see this screen already filled out the next time they sign in).

Step 2 of 4:  Personal Info
Here we capture some demographic information (particularly important for usability testing).  Also, testers can now specify which types testing they want to do (functional, usability, load & performance). Speak multiple languages? Be sure to let us know that too!

Step 3 of 4:  Hardware and Software
As our customers and their apps get bigger and more mature, they’re asking for more specific hardware/software requirements every day.  So add all of your computers, mobile devices, gaming consoles, MP3 players, etc. to your profile and you’ll never miss out on a release again.

Step 3A:  Web and Desktop Devices
Here you can specify your computer details, including your OS, version, language, anti-virus and anti-spyware information. You can add multiple computers and update this information at any time (when you buy a new computer, upgrade your OS, download a new browser or anti-virus program).

Step 3B:  Mobile Devices
As our list of mobile customers expands dramatically, we’re constantly looking for more mobile testers with different device makers, models, wireless carriers and locations. Again, you can add multiple mobile devices and update your profile at any time.  Our new profiles will enable us to find the right testers for each project with no wasted time.

Step 3C:  Gaming and Other Devices
Additionally, specify what types of gaming consoles, MP3 players and digital readers you currently own.

Step 4 of 4:  Experience and Skills
Our testers have a wide range of industry experience, hobbies and skills.  Now you can tell us about your expertise and interests.

This revamped tester profile is a major event in the history of uTest and its community.  There’s much, much more to come, but this new version further extends our lead in the world of crowdsourced software testing.

Have thoughts, feedback or unabashed praise?  We’d love to hear ‘em in the comments below!  uTesters can also join the lively discussion on this topic (and many, many others) in our software testing forums.

Late last night, we launched version 2.5 of the uTest platform.  This new version incorporates a number of infrastructure improvements and bug fixes.  And while some of these things aren’t visible to the naked eye, they are based upon feedback from our customers and testers, and we’re confident they’ll help to improve the overall user experience here at uTest.

Among the more noticeable changes, we’ve improved our support for Google Chrome.  This is among the fastest-growing browsers among our user base, so our platform now supports it.  We’ve also made it easier for new testers and customers to find the answers to their questions with improved FAQs inside the platform.

We’ve also given our project managers several new “behind the scenes” tools and features to help them respond even more effectively to the needs of our customers and testers, as well as to deal with the ever-growing number of releases.

As always, our community deserves a tremendous credit for helping us find and fix a number of bugs as well as for providing us with great feedback.  Have a breakthrough idea for our future product releases?  Join our testing forums and check out our Platform Feedback section.

Tonight we released the version 2.3 of the uTest platform.  There are a lot of fixes and features in this release, but we’re most excited about two new features:

Discussion Threads

Starting today, testing managers have a new tool for working with the uTest community.  With discussion threads, it’s now possible to quickly and easily create a virtual meeting room so they can communicate with their testers and the uTest project manager.

Testers who are working on a project can “meet” in real-time, ask & answer questions and share information with each other about that particular release.  And customers have a great new way to share critical information or requirements while a release is underway.

Discussion threads will enable testing managers to better manage and interact with their virtual testing teams and ultimately create even higher quality apps.

(Click to see a larger screenshot.)

Application Profiling

Another great feature we’re introducing is a tool to help testing managers better match their applications to testers expertise through refined application profiling.  When creating a new release, testing managers will have the ability to classify their application and testing needs.  Our system will use then use this information to automatically identify testers who are more likely to match the release’s exact testing needs.

The drop-down menus will allow testing managers to select:

• Application Type – Web, Desktop, Mobile
• Testing Type – Functional, Usability, or Load & Performance
• Industry Type – Social Networking, Retail, Entertainment/Gaming, Financial Services, etc.
• Audience – Consumer, Business, or Both

This matching process is just the beginning, and coming soon we will release several additional features for both testers and testing managers.  In the coming month, we will be requiring uTesters to update their profiles for even stronger project matching.

In addition to these new features, this release applied fixes to more than 40 bugs (all found by our uTester community).

The word is out!  With the launch of our revamped website, a new “Meet the Testers” application and Tester Forums, last week uTest made significant strides in turning our crowd of 16,000+ QA professionals into an interactive community.

Here’s what the media had to say:

• Jeff Howe, author of best-selling book Crowdsourcing & contributing editor at Wired Magazine, twittered the news to his 2,000 followers calling us out as a Company to Watch.

• Xconomy’s Wade Roush highlights uTest putting a human face on software testing with the new site’s collaboration tools in his story, uTest Site Revamped for Collaboration.

• VCcafe’s Eze Vidra focuses on the “Meet the Testers” app & its drill-down capabilities “into a 16K strong testing community” in uTest Puts a Face on its Crowdsourced QA Community.

Other notable posts at  StickyMinds.com, the Cloud Architect, and MSNBC.com.

Let us know what you think about the new site!