There are more than 250 Android devices, more than 20 active versions of app-supporting iOS devices (the iPhone, iPad and iTouch ranging from 8GB to 64GB models) and countless OS versions, carriers and locations for each one. And that's just two operating systems.
Article Contributed by uTest
Imagine trying to cover that testing matrix with an in-house lab. The cost – both financially and in manpower – would be astronomical. That's where crowdsourced in-the-wild testing comes in.
Crowdsourcing app testing gives you access to thousands of professional testers covering any matrix combination and demographic you can dream up. Need to test a native Blackberry app designed for businessmen who travel a lot in the middle JFK airport? Creating an Android app for mothers of toddlers in South America? Have an iOS app that connects college students from around the world? In-the-wild testing is the only way to adequately test these – and millions of other – situations. Remember, real life users consume apps under:
- Adverse, unpredictable and widely varied environments
- Outdated browsers, plug-ins and third party apps
- Unique hardware and devices
- Imperfect connectivity (both Wi-Fi and mobile carriers)
Leveraging a crowd of global testers allows you to ....
Mirror Real-World Conditions: By moving your testing into the wild – with a handpicked group of testers that match your exact demographics– you get a much clearer picture of how your target users will respond to your application. Essentially, it's like running a beta test, except one that is extremely specific and only includes professional testers.
Identify Fringe Use-Cases: When testing an app in-house, you may identify and test the most popular devices, browsers, and OS versions. But what about all the other devices, browsers and OS versions that are still in use. What about the various third-party applications (e.g. anti-virus, plug-ins, other apps, etc.) that mostly exist on the hardware of your users, and not your QA team. With in-the-wild testing, you get insight into the unusual use cases that can lead to big problems post launch.
Test On-Demand: Unlike most QA projects, in-the-wild testing is designed to be utilized where and when you need it most, requiring very little setup time. This benefits companies whose QA requirements change frequently (usually those adhering to an agile framework). It also drastically cuts back on QA costs since you no longer have to (attempt to) maintain an overwhelming collection of test devices.
Mobile apps and web are only making testing more complicated and keeping up in-house is becoming increasingly burdensome. Moving a portion of testing out of the lab and into the wild with crowdsourced software testing is helping companies like USA TODAY, Tapulous and Microsoft keep up.
Come learn more about crowdsourced software testing with uTest at Crowdopolis.
Click here for the details.