Crowd testing in the Digital Age - why is it growing so fast?

Crowd testing is becoming a popular tool for ensuring application quality in digital organisations. A 2015 Accenture report identified that 43% of European CIO's are either currently using crowd testing or planning to do so in the next two years. That's quite a growth signal! So why is it becoming popular in our 'Digital Age'? Simon Anderson, Assurity Consulting's Head of Platforms, explains.

Crowd testing is a crowd-sourced approach to software testing. Simply put, a crowd platform in the cloud is used to match organisations that need testing services with a crowd of people willing to test.

It's not a new concept – Jeff Howe was credited with the term in a seminal 2006 Wired magazine article. Howe found many examples of organisations engaging a crowd and creating mutual benefit. It didn't take long for someone to apply crowd to software testing. uTest, which began in 2007 and has since been renamed 'Applause', was the first. Many more have followed.

To date, the uptake by organisations has been modest. That's mainly because crowd testing has obvious challenges in security, ensuring crowd behaviour and the vetting of identity and skills. But platforms and processes have improved vastly and the market is seeing this. But there's one more fly in the ointment for crowd testing and it's about how it's been marketed; early strategies positioned crowd as a replacement for in-house and out-sourced testing, rather than a complementary service.      

This story explains a lot! An associate of mine holds a senior quality role in a large organisation with a highly complex IS architecture. He was visited by a rep from a crowd-testing company who stated that all the department's testing needs could be met remotely by the crowd. At that point in the conversation, the representative lost all professional credibility.

The maturing of crowd platforms, services and processes – combined with its positioning as a complementary service – is painting a brighter future for crowd sourcing and crowd testing in particular. Gartner believes its time is now because "the on-demand driven world we live in and the overall impact of consumerization and new technologies affect the requirements on applications' quality and testing in organizations". Gartner sees "crowdsourcing services meeting the increased pressure on the internal quality departments in organizations".

At Assurity, we agree with Gartner, especially in areas of digital testing where traditional teams are not well-positioned to cover testing across multiple devices and operating systems. Just as important is that measuring consumerisation – and the quality of it – is best done through real customer feedback and observation.

The crowd's big advantage, of course, is the diversity of feedback from a specific demographic that matches customer profiles. Talk to a customer-like crowd that owns and uses your applications and you'll build a much more relevant picture of use in real life. You'll see a crowd using your apps in unexpected ways, places and times. You'll learn some things you never knew about.    

The key, of course, is to engage a crowd that best fits your organisation's needs. A crowd does not comprise of ageless, asexual, unemotional automatons. Nor are its members cultureless, without context or place. If your applications are intended for New Zealanders, then you need New Zealanders to test them.

At Assurity, we're focused on developing that New Zealand crowd. We have members who are already professional testers and members who aren't. Most crowd engagements consist of both. We're building a crowd community that helps members learn from and share experiences with each other. We strongly believe that the best and most useful crowds are those who are motivated by more than just financial reward – they're people who want to share knowledge, grow and feel good about helping New Zealand organisations.

In developing our own Crowd Testing service, we ran some pilot engagements, including one for a transformation project for a large IS organisation. Following the engagement, the Digital Test Lead believes that "crowd is not really about 'does it work?', it's more about 'what does our customer think?'". There's much more value in that, of course. And it's why we think crowd testing has a bright future here.