How OpenAI’s New Networking Protocol Aims to Solve AI Bottlenecks

extract

How OpenAI’s New Networking Protocol Aims to Solve AI Bottlenecks OpenAI’s MRC protocol aims to address congestion and failure challenges in massive AI clusters as hyperscalers scale to hundreds of thousands of GPUs. OpenAI and a consortium of tech giants have unveiled a new networking protocol aimed at preventing congestion and hardware failures from stalling massive AI clusters, highlighting how networking has become a major chokepoint in hyperscale AI infrastructure. The company detailed a new protocol called Multipath Reliable Connection (MRC), which distributes traffic across hundreds of network paths simultaneously instead of relying on more rigid routing schemes that can create bottlenecks inside large GPU clusters. OpenAI developed the protocol alongside AMD, Broadcom, Intel, Microsoft, and Nvidia and contributed the technology to the Open Compute Project (OCP). The LLM developer said MRC is already integrated into emerging 800Gb/s network interfaces. The announcement offers one of the clearest technical looks yet at the infrastructure challenges emerging inside frontier AI systems, where a single delayed transfer can stall thousands of GPUs waiting to synchronize during training. “Network congestion, link, and device failures are the most common sources of delay and jitter in transfers,” OpenAI wrote in a technical post announcing the project. “These problems get more frequent, and harder to solve, as the size of the cluster increases.” Tackling AI Cluster Slowdowns Large AI training systems depend on tightly synchronized communication between accelerators. Even small delays inside the network fabric can leave expensive GPUs idle while workloads wait for slower nodes to catch up – a problem infrastructure operators often call the “straggler effect.” OpenAI said MRC attempts to reduce those slowdowns by dynamically spraying packets across hundreds of available paths while rerouting traffic around failures in microseconds. Ron Westfall, networking and infrastructure practice lead and vice president at HyperFrame Research, said the protocol signals a broader shift away from traditional networking architectures designed around static paths and isolated connections. “OpenAI is treating the entire AI fabric as a single fluid system instead of a series of isolated connections,” Westfall said. He said hyperscalers are increasingly prioritizing predictable latency and resilience as AI clusters grow larger. “This transition suggests that as AI clusters scale toward millions of GPUs, the industry is prioritizing a specialized Ethernet-plus architecture designed specifically to overcome the tail-latency and congestion bottlenecks that current networking standards are challenged to optimize,” he said. The protocol relies on multi-plane networking and SRv6-based source routing, which allows network interface cards to encode routing decisions directly into packet headers rather than depending entirely on switch-level routing logic. According to OpenAI, that approach helps bypass congested paths and simplifies rerouting during hardware failures. Ethernet Momentum The announcement also lands amid a broader industry fight over the future of AI networking infrastructure. Nvidia's InfiniBand technology has long dominated large-scale AI training environments due to its low latency and tightly integrated performance. But hyperscalers and cloud operators are increasingly pushing Ethernet-based alternatives as AI deployments scale into larger environments. Sameh Boujelbene, vice president at Dell’Oro Group, said the OpenAI effort reinforces Ethernet’s growing position in hyperscale AI infrastructure. “OpenAI’s MRC work is a strong data point that hyperscalers are leaning harder into Ethernet for AI fabrics, especially as clusters push toward 100,000 to 500,000-plus GPUs,” Boujelbene said. “Historically, ultra-large training clusters were dominated by NVIDIA’s InfiniBand, prized for low latency and tightly coupled performance, but Ethernet has been catching up fast,” she said. Boujelbene said the shift is already showing up in market data. “In 2025, Ethernet sales and shipments to AI back-end networks way surpassed that of InfiniBand,” she said. Westfall said OpenAI’s approach attempts to combine InfiniBand-style performance with Ethernet’s operational flexibility and ecosystem scale. “From my viewpoint, this packet spraying approach provides the losslessness of InfiniBand with the flexibility of a stateless, global IPv6 standard,” he said. AMD echoed that framing in its own announcement supporting the project. “As GPUs and CPUs continue to drive compute, the real bottleneck in scaling AI is the network,” Krishna Doddapaneni, corporate vice president of engineering at AMD’s Network Technology Solutions Group, said in a statement. OpenAI’s decision to contribute the protocol to OCP also signals an effort to push the technology toward broader industry adoption rather than keeping it proprietary. Westfall said the OCP move could help hyperscalers reduce dependence on tightly controlled networking stacks while improving operational resilience inside giant AI factories. “This transition enables the deployment of cost-effective, gigascale Ethernet hardware,” he said. “It can streamline operational management by creating a self-healing network that preserves peak GPU utilization despite hardware failures.” Stargate Scale The networking announcement arrives as OpenAI accelerates its broader Stargate infrastructure push. The company recently said it surpassed 10 GW of secured AI infrastructure capacity ahead of its original 2029 target, adding more than 3 GW over the past 90 days as demand for AI compute continues to surge. That scale is beginning to expose infrastructure pressures that extend beyond GPUs themselves, including power delivery, cooling, network reliability, and synchronization overhead across giant accelerator fleets. OpenAI framed MRC as part of a broader effort to improve utilization efficiency across frontier-scale systems, where idle GPUs can rapidly inflate training costs. Still, the announcement underscores how networking is emerging as one of the central battlegrounds in next-generation AI infrastructure – particularly as hyperscalers push toward clusters large enough that small network disruptions can cascade into major compute inefficiencies.