As the saying goes, an ounce of prevention is worth a pound of cure. For companies that develop public-facing software, this translates into test, test again and test some more before your release your app into the wild. But here’s the dilemma – what if the app you’re testing is top secret? I’m not talking about a new feature like Facebook’s real-time chat, but rather an integration of features between Instagram and Facebook before the acquisition announcement was made earlier this year. If you were the CTO of Facebook, would you have risked testing this integration to anyone outside your company’s firewall?

Let’s say you take the conservative approach and keep a tight lid on the news. On the one hand, you have higher control of top secret information, which should lower risk of any leaks. On the other hand, this may lead to higher risk of launching a buggy app that disappoints your end users. Additionally, you may severely limit your testing coverage and lengthen the QA process if you want to do things right. So what do you do?

While there’s no one-size-fits-all solution, there are several probing questions to help assess your risk quotient: is the risk of launching buggy software to customers higher than a potential leak, what is the likely damage if a leak occurs, what is the risk of a potential leak from internal or external testing efforts (a leak could just as easily stem from inside your own firewall), who can you trust to test your software, is there a trusted vendor that could help and do they accept and enforce NDAs, and how can you achieve good testing coverage?

Case in point: I asked these very same questions to our VP of Product approximately one week prior to uTest’s recent acquisition of Apphance – do we keep testing to our internal team or use our own community to test the integration? Putting my marketing hat on, it’s difficult to stomach the risk of leaked coverage; putting my product hat on, it’s just as difficult to stomach the risk of a botched integration to existing and prospective customers. Long story short, it was a vigorous debate that ended with a simple solution: test the integration with a smaller than normal team of uTesters – all under NDA, and all Gold/Silver/Bronze testers. The results were very pleasing – juicy bugs that our own internal teams were not able to turn up, true collaboration with a small team of great testers from the US and UK, and very importantly…no leaks in customer confidentiality.

So what do you think? Is beta testing worth the risk? If you have any stories and/or insights to share, we’d love to hear them in the comments section below.