Astera Labs Releases Leo 2 CXL Memory Controllers and Leo X Controller for Rackscale Fabric-Attached Memory

extract

One key use case of the Compute Express Link (CXL) standard is supporting the attachment of various forms of DRAM and other memory over the PCIe-based bus. Thanks to CXL’s high-level cache coherency capabilities, the standard allows devices to not only reach into the host’s DRAM, but vice versa as well, which has the potential to enable multiple new ways of organizing and sharing memory within systems and even larger domains. While adding even more memory to a server at a time when RAM shortages have already sent RDIMM prices through the roof may not be ideal, CXL offers a silver lining: RAM does not have to be DDR5. Modern data center processors all use DDR5. Many DDR4 modules are slowly being pulled from retired systems. Marvell’s Structera line offers a CXL DDR4 3DPC controller and has become popular for the line for hyper-scalers who are using CXL to lower the impact of DDR5 supply constraints. Since we did the video for Marvell, that concept has caught fire in the industry. Now Astera Labs is coming up with a newer CXL controller and a lot more. Astera Labs has spent the last several years finding out with its Leo series of smart memory controllers. CXL-based memory expanders have let system vendors add much larger amounts of RAM than current 16 DIMM (8ch x 2DPC) board designs can offer, but until now that focus has been primarily on total capacity. With RAMmageddon, however, the focus is shifting toward economical ways to add more memory to systems and reuse existing memory. For Astera Labs, the company thinks 2026 will be its time to shine. Backed by the launch of a new generation of Leo 2 smart memory controllers, which expand support for DDR4 memory and add CXL 3.2/PCIe Gen6 connectivity, the company believes the market for CXL memory expanders is growing like never before. But just in case that is not enough, the company’s latest batch of Leo expanders also includes a full-on fabric-attached controller, the Leo-X, which lets Leo-based DRAM pools integrate directly into the fabric networks of rack-scale systems, greatly expanding DRAM capacity within those networks. There are quite a few moving parts to today’s announcement, so let us dive right in. Leo 2 E-series and P-series Expand DDR4 Support, Update Features of CPU-attached Expanders First, Astera Labs is launching its second generation of Leo smart memory controllers for CPU-attached memory expanders. The aptly named Leo 2 series brings the company’s basic E-series controller and its more advanced P-series controller the same set of major feature updates. The main driver for both updates is bringing the company’s Leo controllers up to date with the latest-generation PCIe and CXL standards. Whereas the first-generation Leo expanders were PCIe Gen5/CXL 2.x based, the Leo 2 family increments that to PCIe Gen6/CXL 3.2, the latest deployed version of their respective standards, and the one the first host CPUs are now launching with. For CXL memory standards, that change alone is a pretty big deal. The underlying PCIe Gen6 standard doubles the bandwidth available per lane, and by extension per expander, effectively doubling the memory bandwidth each expander can add to a system. This helps mitigate one of the biggest drawbacks of CXL memory expansion, which is that the aggregate PCIe/CXL bandwidth of a server CPU is not as high as its DRAM bandwidth. The CXL 3.x standard also adds some rather useful features on top of that bandwidth improvement, including multi-level switching, direct peer-to-peer memory access, and support for multiple memory expanders per PCIe root port. Altogether, the combination of bandwidth and new features helps to make CXL memory expanders quite a bit more useful than before. With that in mind, Astera Labs has needed to update its Leo controller family not only to use the new protocols, but to put that bandwidth to good use. Double the CXL bandwidth means you need double the DIMM bandwidth to feed an expander, and DDR4 memory is not getting any faster. To that end, the Leo 2 memory controllers have doubled the number of memory channels used on each card, moving from 2 channels to 4 channels. Doubling the channels doubles the bandwidth and, in the process, doubles the total memory each expander can support, with Astera Labs qualifying up to 768GB of DDR4 or 4TB of DDR5 behind each expander. As an aside, Astera Labs seems to be really pushing the envelope on DDR4 capacity here. Among their Leo 2 board designs is a 4-channel, 3DPC board, which is a DDR configuration that is almost never seen in the wild because of the complexities involved. Three DIMMS per channel ultimately allows Astera to mount 12 DDR4 DIMMs into a single expander. DDR5 designs are not quite as lucky, however, as DDR5 is difficult enough to run at just 2DPC. As outlined earlier, the use case for the Leo 2 E-series and P-series expanders is not only to add more RAM to a system, but at this point in time to give data centers a way to do so cheaply. New DDR5 is expensive. High-capacity DDR5 even more so, but DDR4 (and even pre-MRDIMM DDR5) coming off of a retired server is cheap. With the industry as a whole focusing on economics despite the ongoing buildouts (and in particular, the economics of AI tokens), that is a pretty attractive use case. Redeploying large volumes of DDR4 is not without its challenges, however. Even solid-state hardware such as DRAM degrades over time, and most DDR4 memory from old systems is at least half a decade old. To that end, a major focus of the new Leo 2 controllers is their RAS capabilities, which are needed to make DDR4 reuse viable. Among these features is a hardware test engine that can test each DIMM with multiple patterns to make sure it works without fault, and even then Astera is running soft-ECC on top of the DIMMs’ hardware ECC in order to increase how robust the memory pool is and to catch errors that make it past hardware ECC. The ultimate goal is that when a DDR4 DIMM does fault (and in a large-scale deployment, it is a matter of when), the fault can be corrected and the DIMM kept in service rather than immediately being pulled. With regard to features, the Leo 2 E-series and P-series are keeping the same general split in features that the original smart memory controllers had. The Leo 2 E-series will be Astera Labs’ entry-level controller, which is designed to be attached to a single host CPU. Meanwhile, the more advanced Leo 2 P-series controller is meant for memory expanders that are shared between two host CPUs, allowing for systems to dynamically share a pool of memory rather than requiring a fixed allocation per host CPU. Finally, Astera Labs’ marketing angle here unsurprisingly leans heavily into AI markets, as that is where the biggest investments are being made right now, and where some of the greatest memory demand is. The company believes that the new Leo 2 controllers are a good fit for large in-memory databases and for keeping large KV caches local to servers, with the latter in particular being a relatively new use case for CXL memory expanders. In essence, Astera’s argument is a performance one, since a larger KV cache (or rather a large KV cache held in memory instead of going to disk) can allow for faster token outputs. Leo X-Series Attaches Memory Expanders to Accelerator Fabric Networks But while Astera Labs has modest ideas for using the Leo 2 E-series and P-series for KV caching, their biggest play in this space is their third and final new Leo smart memory controller: the Leo X-series. Not content with just attaching more memory to host CPUs over CXL interfaces, Astera has developed a Leo chip that can interface with the network fabrics that back rack-scale AI systems, allowing for memory expanders to become a dedicated memory node within those rack-scale systems and their networks of AI accelerators Technically, the Leo X-series is a synergistic play with another Astera product, the Scorpio line of PCIe switch chips (which we previously looked at here). When Scorpio switch chips are used as part of that network fabric, Leo X-based memory expanders can attach to that fabric by connecting to the Scorpio switch. As a result, the memory expander can be accessed directly by the AI accelerators that make up the network, offering a more direct and potentially performant alternative to going through a host CPU to access more memory. The icing on the cake for Astera, in particular, is that while the Scorpio switches are primarily PCIe switches, the highest-end Scorpio X-series switch chips can also be used in certain proprietary network fabric protocols, making this technology applicable to a wider range of AI accelerator types. Astera does not name which fabrics it supports, but it aspires to target hyperscalers, many of whom either use hardware from a select number of vendors or are developing their own AI accelerators. As a result, Leo X memory expanders can be attached not only outside a rack as part of its scale-out domain, but also within a node as part of the more tightly coupled (and space-limited) scale-up domain. Hardware-wise, the Leo X-series is technically a first-generation accelerator. To that end, Astera has demonstrated a memory expander with four onboard DIMM slots. This is less extreme than what the Leo 2 accelerators can do, but it still allows for either more memory than a current-generation HBM-based accelerator would offer, or a way to add a similar amount of memory to the network at a fraction of the cost of another accelerator. The company has confirmed that this controller supports both DDR4 and DDR5, so server vendors that are choosing cost over total performance can use cheaper DDR4 DIMMs instead of DDR5. In terms of use cases, Astera is going all-in on the Leo X-series as a KV caching solution. Bringing large KV caches closer to accelerators has been a focus across the AI industry lately (e.g., NVIDIA’s Context Memory Storage platform), and for Astera, this is their effort to combine their smart memory controller and PCIe switching technologies into a single solution for the market. As with other efforts, the emphasis is on improving AI accelerator throughput by addressing bottlenecks in the decode phase, which consumes a lot of memory bandwidth and capacity. Officially, Astera is touting that in their testing they found the Leo X-series improved time to first token by as much as 62% and overall token throughput by 22%. That said, Astera’s testing was against a PCIe-based NVIDIA H200 system rather than a newer Blackwell-based system, so it is not a particularly large-scale test. A blog post will go up today detailing their testing and findings. Final Words Astera Labs has been pursuing the memory expander market for several years now, and while the technology has been slow to take off, the company believes that the current AI market and related memory shortage actually play to the company’s strengths. The need for more memory across both general servers and AI accelerators means that more novel approaches are becoming viable in ways they were not before, especially as a way to add memory in a cost-effective manner. To that end, the company is launching its most ambitious lineup of Leo smart memory controllers yet, with the Leo 2 E and P-series, as well as the AI-focused Leo X-series. Between support for the newest PCIe and CXL standards, more memory bandwidth, and more memory capacity, the Leo 2 series is a capable evolution of the company’s CXL smart memory controller lineup. Its fabric-attached Leo X-series is the company’s more ambitious project. Attaching DDR4 and DDR5 memory to the network fabric of an AI accelerator lets Astera Labs not only enter a new market, but also address one of the biggest needs for DRAM expanders. Not to mention, it is a market segment that is more likely to be amenable to the capacity versus bandwidth tradeoffs that these kinds of memory expanders entail. For now, at least, Astera Labs has timing on their side. With CXL 3.x servers now being released and DRAM prices near an all-time high, now is the best window of opportunity yet for their memory expanders. Fittingly, all three of Astera’s new Leo smart memory controllers are now available, allowing potential customers and clients to get started with the hardware right away.