loading
loading
Samsung is showing a CXL memory pool validated with AMD at the OCP Global Summit in San Jose this week, built on its CMM-D CXL memory modules and aimed at agentic AI inference. The companies worked together to validate CXL memory as a way to scale memory capacity and bandwidth alongside CPU core counts, on the premise that agentic workloads drive sharp increases in CPU utilization while memory capacity fails to keep pace with the cores. Samsung’s pooling demo claims up to 38 percent lower time to first token and up to 3.3x higher goodput than static allocation. Samsung’s booth, C41 in the expo hall, splits its memory and storage lineup into Tokenization Solutions, covering HBM4E and LPDDR5X-PIM, and KV-Cache Solutions, covering the CMM-D MD310 CXL module, the SOCAMM2 LPDDR server module, the PM1763 PCIe Gen6 SSD, and the BM1773, which Samsung bills as the first 256TB E3.S QLC SSD. The framing follows the two phases of an LLM pipeline: the prefill and token generation that live in high-bandwidth memory, and the context that has to be held somewhere while the model generates. CXL Memory Pooling Architecture and Multi-Node Sharing The pooling demo runs an AI agent workload on a GPU server against a CXL memory pool built on Dynamic Capacity Device (DCD) semantics, the CXL mechanism that lets a pool hand memory to a host and take it back at runtime. Several LLM instances draw on the pool with elastic allocation, so an instance that hits a context or parameter spike borrows capacity and returns it when the job completes, in place of the usual pattern where each node is sized for its peak and the surplus sits idle. Samsung’s figures for the demo are up to 38 percent lower TTFT and up to 3.3x higher goodput against static allocation; these are vendor results, and we did not run or audit them. The AMD collaboration validates the same approach on AMD’s compute platform, where the stated problem is CPU stalls as agentic workloads scale and memory capacity and bandwidth fail to grow with the core count. Samsung says CMM-D maximizes concurrent agentic AI users by scaling both with the CPU. Neither company has published the server configuration, the CXL switch topology, or interconnect latency and bandwidth figures for the joint setup, and the multi-host sharing side of the architecture is the subject of an October 15 session with XCENA on dynamic pooling, sharing, and processing-near-memory offload. Samsung also has a production data point for the tier the pool creates. Its AI Memory Node with CMM-D runs at Samsung Semiconductor’s US campus, serving an internal AI coding agent and other applications, and Samsung says adding what it calls an L3.5 composable memory tier delivered 5x to 7x KV cache utilization. That work came out of Samsung’s Memory Solutions Lab, its AI platform integrations team, and SGLang. AI Infrastructure Scaling Hojun Shim, corporate vice president and head of the application engineering team in Samsung’s memory business, gave the executive session on October 12 under the title “Scaling Innovation for the AI Era: Keep Your Compute Running at Full Throttle with Highly Scalable Memory.” The argument, as Samsung lays it out across the booth, is that raw memory speed on its own does not keep compute busy as models scale, and that the fix is a hierarchy: high-bandwidth memory for active computation, CXL fabrics that expand capacity on demand, and SSD-backed tiers for the KV cache and model persistence. A second session on October 13, from Jinin So and Youngha Choi of Samsung’s memory business, put that hierarchy in one title, “Tiering the Agent’s Memory: From CXL Pooling to SSD-Backed KV Cache Offloading,” which is the same tiering we tested in our KV cache offload to flash work. Samsung’s related demos cover Flexible Data Placement and a larger NVMe maximum data transfer size inside LMCache for KV cache storage, and a vector database proof of concept that moves distance computation into the SSD. Tokenization On the Tokenization side, Samsung is showing HBM4E, which it began sampling in May and showed alongside an HBM5 model at FMS in August, and LPDDR5X-PIM, which puts a processing-in-memory unit into each DRAM bank. Samsung’s claim for the PIM part is up to 8x the effective bandwidth of standard LPDDR5X and up to 3x the tokens per second on its Edge AI Accelerator, from evaluations Samsung ran on Llama 3.1 8B. The idea is to execute the memory-bound parts of inference inside the DRAM package so less data crosses the interconnect, which cuts power as well as time. Hardware and Architecture Everything on the floor is a demonstration or an existing product: Samsung has not attached a CXL pool controller SKU, availability date, or power rating to the AMD work, and the HBM4E and LPDDR5X-PIM showings carry no product timing beyond the HBM4E sampling already announced. The CMM-D MD310, PM1763, and BM1773 are shipping parts.
document fetched 2026-10-24 09:58 · canonical https://storagereview.com/news/samsung-cxl-memory-pooling-amd-ocp-2026-cmm-d-agentic-ai