Google Cloud says most need upgrades for agentic AI

Google Cloud has outlined its Agentic Data Cloud approach for organisations building AI systems. New research suggests most expect to upgrade infrastructure to support production agentic AI.

The company argues that the main constraint on scaling AI is not model performance, but access to business context and the infrastructure needed to support systems that can query and act across multiple data environments.

Its State of Infrastructure report found that 83% of organisations believe they need infrastructure upgrades to support production agentic AI systems. It also found that 43% of IT leaders identified difficulty integrating with legacy APIs and data sources as the biggest infrastructure gap for agentic AI.

Google Cloud is positioning Agentic Data Cloud as a way to combine data, AI models and operational databases in what it describes as a single system for action. The design relies on close links between compute, networking, storage, analytical systems and operational databases, allowing agents to move between reasoning and execution without repeated manual integration work.

Infrastructure strain

The report's findings reflect wider industry concern over the demands AI agents place on existing systems. Unlike conventional software workflows, an agent may trigger several actions from a single prompt, including browsing, querying and executing tasks across multiple applications and data stores.

This can strain infrastructure layers that were not designed for this type of workload. Google Cloud argues that, without adaptation for agentic AI, data platforms built on top of them will struggle with latency, complexity and cost.

Another finding was that 81% of leaders saw operational complexity and engineering overhead as major unforeseen costs when scaling AI. That points to a practical issue for many large organisations: AI projects can stall because engineering teams spend significant time connecting separate systems rather than deploying finished services.

Data access

A central part of Google Cloud's case is that business context is often trapped in fragmented systems and legacy architectures. That can affect the quality of AI agents' responses if they cannot reach the relevant information or must rely on incomplete data from a single source.

To address this, Google Cloud says its borderless lakehouse model allows access to data across environments using tools including BigQuery and Spanner, built on open standards such as Apache Spark and Apache Iceberg. The aim is to let agents work with data across locations as though it were local, reducing the need to move large datasets into one place.

This matters for large enterprises that have accumulated data across cloud services, on-premises systems and older software estates. Moving everything into a single repository can be costly and technically difficult, so vendors are increasingly promoting architectures that query and use data where it already resides.

Knowledge layer

Google Cloud also highlighted the need for a semantic layer that gives AI systems richer knowledge of enterprise data. It said 36% of leaders cited a lack of specialised high-throughput vector databases for AI grounding as a key infrastructure gap.

The company pointed to its Knowledge Catalog product as a way to aggregate and enrich data in data lakes and support agentic search. In practice, this means extracting meaning from unstructured information and generating semantic context that agents can use when carrying out tasks or responding to users.

For businesses, the issue is not only discovering data, but whether an AI system can safely act on it. Agents that can access live operational systems, including older ERP platforms and third-party CRM tools, need current context and reliable memory so they can complete tasks without reprocessing the same information for each query.

Competitive pressure

Google Cloud's broader message is that AI adoption is shifting from model experimentation to architecture redesign. Many companies have already tested generative AI tools, but turning those experiments into systems that can automate work across finance, customer service, logistics or procurement requires tighter links between data systems and infrastructure.

That has implications for cloud providers, database vendors and enterprise software groups, all of which are trying to define how corporate data should be prepared for AI agents. It also raises spending questions for customers, since infrastructure upgrades can involve new chips, new storage designs, network changes and additional software layers for governance and data access.

Google Cloud argues that the organisations best placed to benefit from agentic AI will be those that can give agents secure, timely and cost-conscious access to the right knowledge at scale. The figures suggest many companies believe they are not yet at that point, with legacy integration, engineering overhead and data grounding still acting as obstacles.

The report's strongest statistic remains that 83% of organisations believe they require infrastructure upgrades to support production agentic AI systems.