IT Brief New Zealand - Technology news for CIOs & IT decision-makers
New Zealand
Securing the leap to autonomous AI

Securing the leap to autonomous AI

Mon, 16th Nov 2026 (Today)
Jan Bee
JAN BEE CISO TeamViewer

In the last few months, the world witnessed a cybersecurity incident involving two AI models. In response, leading AI companies warned about persistent cyber-attacks enabled by AI as the technology enters a new chapter of capability.

This comes as workplace AI is making a transition to autonomous systems. While existing tools largely analyse information or recommend actions, autonomous AI will identify problems and act on them with less human direction.

Incidents of this kind could make employees more hesitant about handing greater control to autonomous tools, potentially stalling adoption as organisations explore their value. Successful adoption will therefore depend on whether organisations can close this confidence gap.

Security teams will play a particularly important role in that process by introducing controls that allow systems to act within a defined scope, while escalating decisions that require human judgement.

Understanding the confidence gap

30% of workers cite security risks or unauthorised access as a leading concern about AI acting without human oversight. As AI systems gain access to workplace data and applications, the consequences of an incorrect or unauthorised action become more significant. For example, an AI system resolving an employee's IT issue could reset the wrong account permissions or grant access to sensitive files. Confidence falls when systems move from recommending an action to taking it independently.

This creates a clear gap between the level of autonomy AI systems are capable of and the level of trust employees are comfortable handing over. One factor behind this gap is a lack of visibility, as employees need to understand how AI systems reach decisions to feel confident in them. They also express needing to intervene, as 70% of employees say they are comfortable with AI acting autonomously if they can step in when needed.

Where human oversight sits

Building that reassurance does not mean requiring people to approve every autonomous action. Instead, the level of human involvement should reflect the potential impact of each task.

IT is likely to be one of the first areas where autonomous systems see broader workplace adoption. Many technology issues follow established processes, making them suitable for autonomous remediation within defined parameters.

For example, an autonomous IT system could identify that a device is running low on storage and apply an approved fix without human input. It could also detect a deteriorating device before it fails, significantly reducing downtime. These lower-risk tasks allow businesses to demonstrate the value of autonomous AI before giving it responsibility for more complex decisions.

Routine tasks may therefore require little intervention, whereas an action involving access to sensitive data may need human approval before it proceeds. Security teams should help define these thresholds so that human oversight is built into decisions with greater potential consequences. 

Make autonomous actions visible

For that oversight to work, organisations need a clear paper trail showing when AI acted and when humans took control. Employees affected by an unexpected change should also have an immediate way to question it.

Take an IT agent authorised to reset access when it detects a locked account. If it resets the wrong account or acts at an unusual time, the employee needs to be able to see why it happened and flag it.

For security teams, visibility must extend beyond these user-facing explanations. Logs should show the resources an agent accessed and the actions it performed. Together, these records provide the auditability needed to investigate anomalous behaviour and establish whether the system operated within its authorised scope.

However, visibility alone cannot prevent an inappropriate action. It must also be supported by clear boundaries governing what the system is permitted to do in the first place.

Set boundaries and monitor behaviour

As AI agents gain permission to interact with workplace systems, security teams need to ensure their access remains proportionate to the task they are expected to perform. Access should be limited to the resources required for the agent's role and reviewed as its responsibilities change. 

These boundaries cannot rely on authentication alone. An AI agent could operate with valid permissions while its behaviour falls outside its intended purpose. Traditional access controls may register the correct identity as authenticated without recognising that the subsequent activity is unusual or inappropriate.

Security teams therefore need to monitor what an autonomous system does after access has been granted. An IT agent designed to handle account access should trigger an alert if it attempts to reach an unrelated application. Activity that deviates from its behavioural baseline should prompt investigation, even when the underlying credentials remain valid.

Security teams should also consider the cumulative effect of autonomous activity. An individual action may fall within policy, while an unusual volume of similar actions could indicate faulty or unexpected behaviour. Risk thresholds should therefore account for changes in frequency and context, with the system escalating decisions to someone with the appropriate authority when those thresholds are reached.

Teams should also be able to reverse any changes AI agents have made, while a defined human owner remains accountable for reviewing its behaviour and responding when something goes wrong.

The next chapter

IT leaders expect 39% of digital workplace services to operate autonomously by 2030, but the success of this transition will depend on whether employee confidence and organisational controls develop at the same pace as AI capabilities.

As AI systems take on more complex and sensitive responsibilities, organisations will need to continually reassess the boundaries governing their activity. Security teams will play a central role in ensuring these controls evolve alongside the technology, allowing businesses to scale autonomy without losing visibility or accountability.

Establishing risk-based controls and meaningful human oversight now will give businesses the confidence to make this transition sec