loading
loading
FLASH VDURA CEO’s Long Live The Short King blog Ken Claffey, VDURA’s CEO, says the SemiAnalysis realization that HBM bandwidth limitations cramp HBM capacity rises beyond 4-hi stacks applies to NVMe SSDs used in KV Cache tiering and that, there too, write bandwidth is more important than capacity. In a blog entitled "Long Live the Short King. Now Look One Tier Down," Claffey writes “Last month SemiAnalysis published “Long Live the Short King: Why 4-hi HBM Wins”. It is about memory stack heights on next-generation accelerators. It is also, though it never says so, the best argument I have read this year for how to size and architect a high-performance durable storage tier.” He says that HBM 4/4e stacks range in capacity, from 4-hi to 12-hi, but the connection pin count, 2,048, is static and means that, whatever the capacity, the bandwidth is the same; “Stack height buys capacity and nothing else.” And capacity that can’t use the available IO bandwidth is stranded capacity. Claffey says: “The same principle governs flash. Bandwidth is a property of the device, set by whichever limit binds first, the interface, the controller, or the media itself. Capacity is a property of what sits behind it. They scale independently, and the premium tier in any hierarchy is premium for its bandwidth, not its capacity.” “An HBM cube gets 2,048 I/Os whether it is 4-hi or 12-hi. A PCIe Gen5 x4 NVMe drive gets its roughly 14 GB/s [read bandwidth] whether it is 7.68 TB or 61.44 TB. In both cases, the bandwidth-to-capacity ratio collapses as capacity grows, and every terabyte past the point where bandwidth binds is bought at premium-media prices for value a cheaper medium could have supplied.” He cites Kioxia’s CD9P PCIe gen 5, TLC SSD as an example; And points out: “Eight times the capacity. Identical write bandwidth, and nine percent less read. The drive does not get faster as it gets bigger, because none of what limits it got bigger; only the NAND behind it did.” Put it this way: “Stranded HBM capacity is capacity you paid for and cannot turn into tokens. Stranded SSD performance is the mirror image: speed you paid for and cannot fully use. It matters because SSD terabytes cost 14x disk drive TBs; serious money is at stake here. This is a 6,000+ word blog crammed full of detailed argument to justify his case that the Semi-Analysis HBM realization applies to NVMe SSDs used in KV Cache-type workloads as well. Okay. Accept that. Then buy NVMe SSDs for this workload with bandwidth as a priority, not capacity, and tier down to external disk drives. Claffey supplies a formula for the number of SSDs needed, saying: “SemiAnalysis sized HBM in two passes. Bandwidth first: how many stacks to hit the throughput and interactivity target? Then capacity: does what those stacks hold cover the model? If it does, height beyond that is money. “A high-performance durable storage tier sizes the same way.” Here is his formula; N = R ÷ (d × E) - N is the number of drives you buy. - R is the bandwidth you require. - d is what one SSD delivers. - E is the fraction of d your architecture actually gets through to a client. "Everything else follows from N: the capacity you end up owning is N times the drive capacity, the flash spend is N times the drive price, and the cost of a delivered gigabyte per second is the drive price divided by d × E." He asserts: “R is set by your accelerators. d is set by whoever made the drive, and everyone is buying from the same short list of suppliers. E is set by the architecture, and it multiplies all of the rest. Halve it and you buy twice the drives, carrying twice the capacity nobody asked for, drawing twice the power. Buy a larger drive instead and you raise the capacity term without raising d, which improves exactly one number on the quote and worsens every other one.” Claffey outlines a 5-step SSD buying guide for these workloads; - Start with bandwidth, both directions. - Then count devices, not terabytes. - Then take the smallest drive that gets you to N. - Then check what capacity came along. - Then put the rest downstairs - in a hard disk drive capacity tier. But, he notes, “the fall from flash to disk [performance] is not a small multiple. It is orders of magnitude. So the whole strategy stands or falls on the tiering, and not on whether you have some, but on whether it is fit for this workload. … The capacity tier has to be built for throughput, not for dollars per terabyte alone. A disk tier specced like an archive (too few devices, each one as large as possible) cannot absorb what a flash tier hands it, and that is a purchase decision you do not get to revisit later.” He hammers this point home: “The flash tier is a buffer, and a buffer only works if it empties as fast as it fills.” VDURA’s parallel file system, with built-in S3 support can, he says, ingest data from the SSD tier and feed it with data, both fast enough to keep accelerator activity high, and also scales performance as capacity ramps up. He finishes his blog by saying: “The short king thesis is right. It is just not finished. It stops at the edge of the accelerator package, and the logic does not.” Put aside a quiet 30 or 40 minutes and read his blog. It’s provocative and quite complex, and even if you disagree the process of justifying or rationalizing your disagreement will be worthwhile.
document fetched 2026-10-28 02:51 · canonical https://blocksandfiles.com/flash/2026/10/27/vdura-ceos-long-live-the-short-king-blog/5305718