AI's Data Centre Overhaul Forcing a Network Rethink

Data centres are at a structural turning point as artificial intelligence reshapes not just compute demand, but the very assumptions underpinning networking and infrastructure design.

What was once an incremental evolution of cloud architecture is now a wholesale redesign of how data moves and how systems scale. At the centre of this shift is a simple but increasingly unforgiving reality: AI workloads do not behave like traditional cloud applications.

Rather, they are continuous, tightly synchronised, and massively parallel. Instead of isolated requests and responses, modern AI training involves thousands of GPUs exchanging data simultaneously, often across racks and even across facilities.

The result is a network environment where performance is no longer a background concern, but has a direct impact on business performance.

The industry has already lived through two major networking eras. The "north-south" model of early cloud computing was built around user traffic entering and exiting data centres, with relatively modest internal communication needs.

That evolved into an "east-west" world driven by microservices, where server-to-server traffic inside the data centre became dominant. Now AI has pushed this pattern into overdrive. Internal traffic is no longer just frequent but also constant and highly sensitive to delay. In effect, east-west traffic has become the entire workload.

'Pushing the piano'

The analogy often used inside the industry is that of "pushing pianos around the office." Instead of individual tasks moving independently, thousands of components must co-ordinate large, heavy data movements in lockstep. If one participant falters, the entire group slows.

Even small inefficiencies can cascade into queue buildup, GPU idle time, and significantly higher training costs. With accelerator clusters costing millions of dollars and operating continuously, every second of inefficiency becomes directly measurable in financial terms.

This is driving a rapid shift in architectural requirements. Oversubscription ratios that were once comfortably designed at 3:1 or 5:1 are now collapsing toward 1:1 models in AI clusters, where every GPU must communicate with every other GPU simultaneously.

Bandwidth demands are rising just as sharply, with 400Gbps Ethernet becoming baseline, 800Gbps deployments accelerating, and 1.6Tbps already on the horizon. Traditional "best effort" networking is increasingly viewed as insufficient for workloads that require predictable, near-lossless performance.

An evolving technology mix

To meet these demands, data centre operators are adopting technologies such as RDMA over Converged Ethernet (RoCE), along with congestion management mechanisms like Explicit Congestion Notification (ECN) and Priority Flow Control (PFC). These tools are designed to minimise buffering and prevent packet drops in environments where even microsecond-level delays can propagate into system-wide slowdowns.

At a higher level, the balance of compute itself is shifting. The GPU is becoming the primary unit of computation, while the CPU increasingly plays a co-ordinating role. This inversion is reshaping how servers are designed and how networking stacks are integrated.

Technologies such as SmartNICs and data processing units are offloading networking, security, and telemetry tasks closer to the fabric, reducing CPU overhead and improving synchronisation across distributed workloads.

These architectural changes are also driving physical transformation. Power densities of 50 to 150 kilowatts per rack are pushing air cooling beyond its limits, accelerating the adoption of direct-to-chip liquid cooling and immersion systems.

Networking hardware is undergoing a similar transition. Electrical interconnects are reaching their practical limits, accelerating a shift toward optical technologies. Co-packaged optics, which bring photonic interfaces closer to compute units, are reducing both latency and energy consumption, while enabling higher bandwidth density within constrained physical footprints.

The changing role of the network

In this emerging environment, the network is no longer passive but is becoming an active participant in workload management. Model updates, fine-tuned weights, and inference optimisation data are now treated as first-class traffic types, alongside traditional application data. 

And the network is not just a neutral transporter of data. Production-grade AI data centres need to run on a lossless, high-performance network fabric – supporting low latency, preventing oversubscription, and able to deliver east-west bandwidth at scale. Organisations aren't looking for a data centre network, they need an AI fabric, where the network is part of the application itself.  

Taken together, these shifts point to three structural changes that define the AI infrastructure era. First, the network now sits directly in the critical path of compute performance and financial efficiency.

Second, networking systems are becoming adaptive, continuously optimising in response to workload behaviour rather than static configuration.

Third, compute and networking design are converging, with hardware and software increasingly co-designed rather than independently optimised.

The implication is that future competitiveness will not be determined solely by faster processors or larger clusters, but by how effectively compute and networking are integrated into a unified architecture.

Data centres are evolving into tightly coupled, GPU-dominated systems supported by lossless, high-bandwidth optical fabrics, while the edge becomes a distributed extension of the same intelligence layer.

The trajectory is clear: infrastructure is shifting from general-purpose computing environments into highly specialised AI factories. What once looked like incremental upgrades in speed and capacity is now a fundamental redesign of the digital economy's physical backbone.