AI success takes more than a promising pilot project

Organisations are flying ahead, launching AI projects at a pace never seen before by the technology industry. While other revolutionary technologies that have changed how we work have come before, AI is impacting work, and the broader society, in ways we could never imagine.

While proofs of concept, prototypes and larger projects are forging ahead, there is one area of successful project management that is languishing: governance.

Thoughtworks' CIO Jessie Xia says the nature of AI lends itself to moving faster than other technologies that came before.

"A lot of AI tools are available and provide flexibility. But the normal pace of organisations to put governance together is slower than what a business wants and the speed people move. Our most recent research found that about 88% of the AI adoption is happening faster than a governance structure can adopt."

There are several reasons behind this, Xia said. One is that AI is more distributed than other technologies. With AI being widely accessible, there are multiple functions where leaders are involved in AI decisions. Security and data, legal and risk compliance teams are heavily involved in AI decisions. But many organisations don't have clear decision-making arrangements in place.

"Governance is part of how AI is designed and deployed," said Xia. "I think many organisations try to establish policies. However, those policies are on paper. They need to build those policies into the architecture and process controls. They need to build in traceability and establish clear accountabilities."

Even with good governance in place, moving from a trial to production is a challenge.

The typical obstacles we see are security, data quality and integrations with existing systems. A business department might want to build something very quickly by themselves. But then they struggle to go to production," said Xia.

These observations are backed by Thoughtworks' survey with about a third of respondents reporting that data quality and the integrations are issues.

"Even when you successfully demonstrate the application can work it doesn't necessarily mean a solution is secure and ready to be used in the enterprise. A pilot can work in controlled settings, but when you build something integrated with other systems, especially for large organisations, you need to look into how the new applications integrate with the current infrastructure and other systems," said Xia.

Another key consideration is ROI and the ownership economics. Careful consideration needs to be given to the outcome being sought and the governance and compliance. All these factors present challenges for taking pilot projects into production and to create value for the organisation.

The distributed nature of AI and its governance demands that organisations refine corporate structures and roles. In Thoughtworks' case, Xia said the company has a Chief AI Officer as part of the leadership team to ensure all the moving parts, from software development and governance to infrastructure and security are coordinated. While it is difficult to centralise everything into one role, it is important to ensure there is coordination.

In preparing for the future, Xia said AI literacy will become increasingly important. But that won't be enough. Critical thinking and judgement will become more valuable as the roles of people change.

"We're already seeing AI doing more and more execution work such as coding and process automation. People need to really understand what roles they are playing here and use critical thinking to challenge what AI is doing and be more innovative. People will need to evaluate whether AI tools deliver biased responses. People will need to continuously uplift their capability for judgment and critical thinking to realise the limitations of AI and to define how AI and humans should interact," she added.

Just as workers in the 1990s needed to develop computer literacy, today's workers will need to develop AI literacy. The cohorts that enter the workforce over the next decade will arrive with AI literacy. But they will need to develop critical thinking, collaboration and other human skills to ensure they remain valuable.

When it comes to the local picture, Xia said that while Australian organisations are doing well, most offer 'qualified' maturity when it comes to AI governance.

"Australian organisations are prioritising coordination across business units. And the research found 55% are embedding dedicated technology leads with the business units. We also saw 48% establishing formal AI governance and operating models with 38% standardising software and digital transformation across business units," said Xia.

With AI on every corporate agenda, Xia offers some important advice.

"Establish clear accountability before you scale and know who owns the business outcomes. Put identity, data access, monitoring and traceability into the system and automate these where you can to make governance easier. Finally, we need to measure value and redesign our workforce and ways of working. Invest in workforce capability and redesign workflows and processes so people understand where AI adds value and where human judgment remains essential."

Organisations want to move faster with AI, but they need foundations in place to move beyond experimentation to production. Clear accountability, governance built into systems and a focus on measurable business value must sit alongside investment in people and new ways of working. Turning the promise of AI into measurable and sustainable value will rely as much on human judgement as on the technology itself.