Broadcom and FuriosaAI Bet on Ethernet AI Fabrics

extract

Broadcom and FuriosaAI Bet on Ethernet AI Fabrics Broadcom and FuriosaAI are building a rack-scale inference platform aimed at shifting AI infrastructure toward Ethernet fabrics, chiplets, and power-efficient token generation. Broadcom and South Korean AI infrastructure startup FuriosaAI are building what they describe as a rack-scale inference platform designed to push AI infrastructure beyond the GPU-centric architectures that currently dominate the market. The partnership, announced Wednesday, pairs FuriosaAI’s third-generation Tensor Contraction Processor architecture with Broadcom’s networking, packaging, and interconnect technologies to build a multi-die chiplet inference system aimed at hyperscale AI deployments. The companies framed the effort as more than a silicon collaboration. The platform combines Furiosa’s inference architecture with Broadcom Ethernet fabrics, PCIe technologies, advanced packaging capabilities, and AI infrastructure IP to scale inference clusters across thousands of nodes. Alex Liu, SVP of product and business at FuriosaAI, told Data Center Knowledge the effort was “a strategic partnership that moves beyond a traditional ASIC partnership,” combining Furiosa’s accelerator architecture with Broadcom packaging, Ethernet switching, and PCIe technologies to build infrastructure capable of scaling “compute clusters to thousands of nodes.” The announcement arrives as AI infrastructure operators increasingly confront a new problem: inference workloads are beginning to scale faster than the training environments that initially drove the generative AI boom. From Servers to Rack-Scale Infrastructure The announcement extends a broader strategy Furiosa has been building around vertically integrated inference infrastructure. Earlier this year, the company introduced its own RNGD inference servers built around dense, air-cooled deployments optimized for enterprise and private cloud AI environments. At the time, Furiosa CEO June Paik framed the effort as part of a larger push to reduce dependence on NVIDIA’s software ecosystem. “The challenge is for us to replace this CUDA engine with our own software stack,” Paik told Data Center Knowledge in an earlier interview. The Broadcom partnership expands that strategy from single-server optimization into rack-scale networking and cluster architecture. Inference Changes the Infrastructure Equation The announcement also reflects a broader shift underway inside AI infrastructure as inference workloads begin diverging from the training systems that fueled NVIDIA’s rise. While large-scale model training still depends heavily on tightly coupled GPU clusters and proprietary interconnects such as NVLink, operators increasingly face a different set of constraints when deploying inference infrastructure at production scale: power density, networking efficiency, memory bandwidth, latency, and token throughput. Ron Westfall, VP and practice lead for networking and infrastructure at HyperFrame Research, said large-scale inference is shifting AI infrastructure priorities away from the requirements that shaped GPU-heavy training clusters. “Large-scale inference shifts the bottleneck toward optimizing TCO, memory bandwidth, and power consumption per token,” Westfall said. “Inference performance is no longer defined solely by raw compute,” Charlie Kawwas, president of Broadcom’s Semiconductor Solutions Group, said in the release. “It is increasingly a function of data reuse and communication efficiency across servers and racks.” That framing places networking and data movement at the center of next-generation inference design. Westfall said the partnership reflects growing industry emphasis on networking efficiency as AI deployments scale beyond tightly coupled training systems. “Optimizing networking efficiency and rack-scale interconnectivity is now just as vital to inference economics as raw silicon performance,” he said. Ethernet Fabrics Move to Center Stage The companies said Furiosa’s third-generation accelerator will use a 2nm compute die paired with dual-layer HBM4/4E memory and Broadcom packaging technology to integrate multiple silicon dies into a single system-on-chip platform. Broadcom’s Ethernet and PCIe technologies will provide the high-bandwidth rack-scale connectivity required to scale large inference clusters, according to the companies. The architecture also signals growing industry momentum behind Ethernet-based AI infrastructure as vendors search for alternatives to proprietary GPU fabrics. Broadcom has increasingly positioned itself as a supplier of the networking, switching, and interconnect infrastructure underpinning large AI clusters, particularly as hyperscalers pursue custom accelerators and heterogeneous compute environments. The Furiosa partnership also fits into Broadcom’s broader AI infrastructure strategy. Earlier this year, Broadcom CEO Hock Tan said the company saw a path to delivering more than $100 billion in AI chip revenue by 2027, driven by hyperscaler demand for custom silicon and diversified AI architectures beyond traditional GPU deployments. Westfall said the industry is beginning to optimize entire racks rather than individual accelerators as operators confront power, cooling, and scaling constraints. “Optimizing large-scale token generation is no longer just a silicon challenge, but a holistic systems-engineering problem driven by the network itself,” he said. Furiosa argues inference infrastructure requires different architectural assumptions than GPU-based training systems. “By focusing on high-bandwidth data movement rather than the thread management required by GPUs, the chip will deliver higher performance-per-watt and greater token density than state-of-the-art GPUs,” the company said in the release. That claim aligns with Furiosa’s earlier positioning around dense inference deployments built to operate inside conventional air-cooled data centers rather than the increasingly power-hungry liquid-cooled GPU environments emerging around frontier AI training clusters. Building for the “Token Factory Era” The partnership builds on Furiosa’s second-generation RNGD inference accelerator, a 180W PCIe-based chip now in mass production on TSMC’s 5nm process, according to the company. Furiosa said RNGD has been deployed by Samsung SDS and LG AI Research for large language model and agentic AI workloads in standard air-cooled data centers. Paik described the new system as infrastructure designed for the “token factory era,” a phrase increasingly used across the AI sector to describe industrial-scale inference operations serving massive numbers of AI-generated responses. Agentic AI workloads are generating “massive volumes of tokens,” shifting infrastructure bottlenecks away from the models themselves and toward systems capable of scaling across thousands of interconnected nodes, Liu said. The company also emphasized software portability as part of the platform strategy. Furiosa said its SDK uses a compiler-based approach that automatically maps PyTorch code onto its silicon architecture, avoiding the extensive kernel-level optimization often required for GPU deployments. The partnership also highlights how AI infrastructure competition is increasingly moving beyond raw accelerator performance and into the economics of moving data efficiently across racks, networks, and power-constrained facilities.