loading
loading
At OCP 2026, I was fortunate to share an Uber ride from the hotel to the convention center with Microsoft’s project manager for the Maia 300, the company’s new in-house AI acceleration platform. I promised that we would cover it, so it is time, because it is a neat platform. OCP Summit 2026 was much larger than it has been, but keeping with tradition, Microsoft had an awesome booth. Microsoft Maia 300 Shown at OCP 2026 A few weeks before OCP 2026, at Hot Chips 2026, Microsoft did a deep-dive on the Maia 200. For OCP, it showed the next-generation chip, the Maia 300. This has eight HBM4 stacks for 288GB of memory and 18TB/s of memory bandwidth, and 3.2TB/s of scale-up bidirectional bandwidth. Microsoft says that it fits 96 Maia’s into a rack and roughly 12K in the scale-up domain. Instead of just bringing a chip to the show, they brought an entire 4-accelerator node, which is what makes OCP awesome. Just to give folks a look, here is the overhead view of the node from a stock photo. One thing you can clearly see is that the 950W TDP accelerators are covered by a liquid-cooled cold plate. You can also see what look like MCIO x16 connectors on the board, but more interestingly, there are many cabled connections. Our best guess is that these are UltraPass connectors that help scale within the node. Microsoft has a pretty neat PCIe topology and extensive fabric connectivity throughout the chassis. Another item we noticed is that this is not an all-liquid-cooled design. The CPU, SSDs, OCP NIC 3.0 devices, and many components around the AI accelerators are cooled by a massive fan partition that spans the chassis. At the rear, it connects to busbar power, blind mate liquid cooling, and its scale-up network. One thing is very noticeable here. Whereas NVIDIA is designing its Vera Rubin NVL72 nodes and infrastructure to use fewer cables, Microsoft is all-in on cables, as even each accelerator is on its own PCB with multiple cables, and there are many PCBs in this system. You may have also noticed that this is more like a 1.5x standard OU width, but not quite the double-wide width of the AMD Helios. When you are customizing this much inside the server, from designing the ASICs, perhaps customizing the racks as well is not a big deal. Final Words Microsoft makes some really cool stuff, and this is not an exception. You may have seen the Massive Microsoft Azure HBv4 AMD EPYC Genoa Heatsink years ago, where even the custom-designed heatsinks are just cool. Of course, the components have well over 4kW TDP, and a rack has 24 of these trays, so I know STH readers love seeing new AI accelerators with lots of bandwidth. As an aside here, when I saw the 288GB HBM4 and 950W specs, a part of me wondered what would happen if there was ever a way to have something like the DGX Station (e.g., this Gigabyte or this MSI, and the Supermicro review will be next week) with these accelerators. Even though these are six-figure tower systems, from what I have heard, demand is crazy. That would probably take Microsoft deciding it wanted to build a developer ecosystem and start selling Maia, but it would be interesting, especially if it hit a lower price point.
document fetched 2026-10-28 02:51 · canonical https://servethehome.com/microsoft-maia-300-shown-at-ocp-2026