Story image

Crowd testing in the Digital Age - why is it growing so fast?

11 Jul 16

Crowd testing is becoming a popular tool for ensuring application quality in digital organisations. A 2015 Accenture report identified that 43% of European CIO’s are either currently using crowd testing or planning to do so in the next two years. That’s quite a growth signal! So why is it becoming popular in our ‘Digital Age’? Simon Anderson, Assurity Consulting’s Head of Platforms, explains.

Crowd testing is a crowd-sourced approach to software testing. Simply put, a crowd platform in the cloud is used to match organisations that need testing services with a crowd of people willing to test.

It’s not a new concept – Jeff Howe was credited with the term in a seminal 2006 Wired magazine article. Howe found many examples of organisations engaging a crowd and creating mutual benefit. It didn’t take long for someone to apply crowd to software testing. uTest, which began in 2007 and has since been renamed ‘Applause’, was the first. Many more have followed.

To date, the uptake by organisations has been modest. That’s mainly because crowd testing has obvious challenges in security, ensuring crowd behaviour and the vetting of identity and skills. But platforms and processes have improved vastly and the market is seeing this. But there’s one more fly in the ointment for crowd testing and it’s about how it’s been marketed; early strategies positioned crowd as a replacement for in-house and out-sourced testing, rather than a complementary service.       

This story explains a lot! An associate of mine holds a senior quality role in a large organisation with a highly complex IS architecture. He was visited by a rep from a crowd-testing company who stated that all the department’s testing needs could be met remotely by the crowd. At that point in the conversation, the representative lost all professional credibility.

The maturing of crowd platforms, services and processes – combined with its positioning as a complementary service – is painting a brighter future for crowd sourcing and crowd testing in particular. Gartner believes its time is now because “the on-demand driven world we live in and the overall impact of consumerization and new technologies affect the requirements on applications’ quality and testing in organizations”. Gartner sees “crowdsourcing services meeting the increased pressure on the internal quality departments in organizations”.

At Assurity, we agree with Gartner, especially in areas of digital testing where traditional teams are not well-positioned to cover testing across multiple devices and operating systems. Just as important is that measuring consumerisation – and the quality of it – is best done through real customer feedback and observation.

The crowd’s big advantage, of course, is the diversity of feedback from a specific demographic that matches customer profiles. Talk to a customer-like crowd that owns and uses your applications and you’ll build a much more relevant picture of use in real life. You’ll see a crowd using your apps in unexpected ways, places and times. You’ll learn some things you never knew about.     

The key, of course, is to engage a crowd that best fits your organisation’s needs. A crowd does not comprise of ageless, asexual, unemotional automatons. Nor are its members cultureless, without context or place. If your applications are intended for New Zealanders, then you need New Zealanders to test them.

At Assurity, we’re focused on developing that New Zealand crowd. We have members who are already professional testers and members who aren’t. Most crowd engagements consist of both. We’re building a crowd community that helps members learn from and share experiences with each other. We strongly believe that the best and most useful crowds are those who are motivated by more than just financial reward – they’re people who want to share knowledge, grow and feel good about helping New Zealand organisations.

In developing our own Crowd Testing service, we ran some pilot engagements, including one for a transformation project for a large IS organisation. Following the engagement, the Digital Test Lead believes that “crowd is not really about ‘does it work?’, it’s more about ‘what does our customer think?’”. There’s much more value in that, of course. And it’s why we think crowd testing has a bright future here.

Three ways to achieve data security whilst enabling BYOD
"A mobility strategy is now more important than ever before, that said, selecting the right one is often no small task."
Mobile Infrastructure market sees fastest growth since 2014
The report from Dell’Oro shows that while the vendor rankings for the top three vendors remained unchanged with Huawei, Ericsson, and Nokia leading.
HPE unveils AI-driven operations for ProLiant, Synergy and Apollo servers
With global learning and predictive analytics capabilities based on real-world operational data, HPE InfoSight supposedly drives down operating costs.
Deloitte bolsters AWS offerings with CloudinIT
“By joining forces we can help even more organisations adopt cloud technologies and put their customers at the heart of their digital agendas.”
How IoT and hybrid cloud will change in 2019
"Traditional VPN software solutions are obsolete for the new IT reality of hybrid and multi-cloud."
Enterprises to begin closing their data centres
Dan Hushon predicts next year companies will begin bidding farewell (if they haven't already) to their onsite data centres.
Citrix acquires micro app platform Sapho
Sapho’s micro applications improve employee productivity by consolidating access to tools, activities and tasks in a simple and unified work feed.
HPE expands AI-driven operations
HPE InfoSight extends select predictive analytics and recommendation capabilities to HPE servers, enabling smarter, self-monitoring infrastructure.