When One Datacenter Is No Longer Enough

Training the largest AI models is no longer simply a question of adding more GPUs. As clusters grow, power availability is becoming a hard physical constraint, forcing infrastructure teams to consider how a single training workload can operate across multiple data centres.

That creates a very different networking problem. Traditional datacenter interconnect was designed to move traffic between sites. AI training demands something more exacting: huge, synchronous flows, minimal packet loss and tightly coordinated communication between GPUs. If one part of the cluster stalls, the impact can ripple across the entire job.

In this interview, The Register’s Tim Phillips speaks with Rakesh Chopra, SVP, Silicon and Systems Architecture, Cisco Fellow, about what Cisco calls the “Scale-Across” imperative and why it believes distributed AI infrastructure requires a new approach to routing.

Rakesh explains how the demands of AI are changing the role of the network, from simple transport to an integral part of the compute system. He discusses the challenge of making geographically separate facilities behave more like a single deterministic machine, and why buffering, high-speed coherent optics and tightly integrated silicon matter when workloads span long-distance fiber links.

The conversation also looks inside the cluster itself. Rakesh outlines how Cisco’s Silicon One architecture and Intelligent Collective Networking are designed to manage synchronized bursts of GPU traffic, reduce bottlenecks and improve job completion time. Power efficiency is another central theme, including the trade-off between network consumption and the energy available to GPUs.

Security and longevity also come into focus. The interview covers hardware-accelerated MACsec and IPsec, as well as the role of programmability in helping infrastructure adapt as AI workloads continue to evolve.

For infrastructure and datacenter leaders planning for larger, more distributed AI environments, the discussion offers a practical view of the networking challenges that emerge once one site is no longer enough.

SPONSORED POST: Why training AI across multiple sites is creating a new networking challenge

Dell Adds Data Context, Prep, And Storage Features To Its AI Data Platform

AI Spending Is The Honey Badger Of Budgets

Micron’s Numbers Show Why HBM4 Memory Is Worth 15X Its Weight In Gold

Build The Right AI Factory For Your Needs: Partner For Success

Optimization Is The Key Challenge For Quantum Computing

Nvidia Wraps Security Layer Around Agentic AI To Stop Rogue Behavior

AMD Buys World Labs For $8.2 Billion To Get AI Model Cred

IonQ: There Is One CPU To Rule Error Correction Decoding

Improving Quantum Error Correction By Meshing Surface Code With IBM’s Heavy-Hex Architecture

Vector Search Is Now A Data Type, But The Money Math Has Not Gone Away

Sovereign AI Overcomes Compliance Challenges And Feeds Innovation

Accelerated Server Sales Are Themselves Accelerating

The GenAI Boom Accelerates And Transforms Ethernet Switching

Bull Beats Out HPE For Next-Gen Lumi AI Supercomputer

Infleqtion And Nvidia Reduce Physical-to-Logical Qubit Ratio in Quantum Computers

How To Smash The Memory Wall Plaguing High Performance Systems

Operationalize AI At Scale With HPE And Nvidia

Despite Hefty Capital Investments, The Cloud Starts Paying Off For Oracle

Broadcom Rides Rocketing Trend For Custom AI Accelerators

Startup d-Matrix Will Pair Its Raptor Memory-Based XPU To Nvidia Rackscale Iron

HPE Sees No Customer Pushback On Rising Datacenter Equipment Costs

Dell Says AI Will Drive 75 Percent Of Datacenter Demand By 2030

Nvidia Expands Its Open Source AI Presence With $12.9 Billion Hugging Face Buy

Optics Still Driving Marvell’s AI Business More Than Custom Chips

VMware Intros Private AI Cloud, AI Factory As Workloads Shift To On-Prem

In The Long Run, Nvidia NVSwitch Is The InfiniBand Of Scale Up AI Networks

Peeling Apart That Supposed $120 Billion Chip Deal Google Inked With Marvell

The Network Is More Of Nvidia’s Computer Than It Ever Was At Sun

Our Websites

Your Privacy

Cookies Policy Privacy Policy Ts & Cs Do not sell my personal information Your Consent Options

Archives 10 years of TNP