KIOXIA CM9-R 15.36TB Review: BiCS8 Flash Hits Full Speed at Low Queue Depths

extract

The KIOXIA CM9-R is the first enterprise SSD family built on eighth-generation BiCS FLASH, and the 15.36TB E3.S model in our lab is the volume sweet spot of the range: 3,400K random read IOPS and 14,800 MB/s of sequential read from a drive with a 25W typical active rating. Gen5 enterprise drives routinely push into the high 20s to hit their headline figures, and KIOXIA’s pitch for the CM9 generation is that BiCS8’s CBA (CMOS directly Bonded to Array) design delivers top-of-class throughput without the power tax. The CM9-R is the 1 DWPD read-intensive arm of the family; a 3 DWPD CM9-V mixed-use line runs alongside it. Capacities span 1.92TB to 30.72TB in E3.S, with a 61.44TB flagship reserved for the 2.5-inch U.2 version. Below 7.68TB, the series carries BiCS generation 5 flash; the 7.68TB, 15.36TB, and 30.72TB models are where generation 8 lives, and our 15.36TB review unit (KCM9XRJE15T3) sits at the exact capacity point where the series posts its best write specs: 11,000 MB/s sequential write and 540K random write IOPS, both series highs shared with no other capacity. The drive is PCIe 5.0 x4 with dual-port x2 support for high-availability topologies, NVMe 2.0 and NVMe-MI 1.2c compliant, and supports the OCP Datacenter NVMe SSD v2.5 specification (not all requirements). Power-loss protection and end-to-end data protection are standard, with SIE, SED, and FIPS 140-3 SED security variants available in the model matrix. KIOXIA CM9-R Specifications Testing Background and Comparables We use a Dell PowerEdge R760 running Ubuntu 22.04.2 LTS as our test platform for all workloads in this review. Equipped with a Serial Cables Gen5 JBOF, it offers wide compatibility with U.2, E1.S, E3.S, and M.2 SSDs. Our system configuration is outlined below: - 2 x Intel Xeon Gold 6430 (32-Core, 2.1GHz) - 16 x 64GB DDR5-4400 - 480GB Dell BOSS SSD - Serial Cables Gen5 JBOF - NVIDIA L4 Drives Compared - KIOXIA CD9P-R 7.68TB (1 DWPD) - Sandisk DC SN861 7.68TB (1 DWPD) - Solidigm PS1010 7.68TB (1 DWPD) - Solidigm PS1030 12.8TB (3 DWPD) - Micron 7600 MAX 6.4TB (3 DWPD) - Micron 9550 MAX 12.8TB (3 DWPD) - Micron 9550 Pro 7.68TB (1 DWPD) The pairing with KIOXIA’s own CD9P-R matters most: the CD9P-R showed what BiCS8 does at data center positioning, and the CM9-R is the enterprise flagship version of the same flash. FIO Performance 128K Sequential Write (IODepth 16 / NumJobs 1) The CM9-R opened its FIO run with a strong steady-state 128K sequential write result: 8,668.1 MB/s at 230.4 µs, third in the group behind only the Micron 9550 MAX (10,957.9 MB/s) and 9550 Pro (10,354.6 MB/s), and comfortably clear of the rest of the field, which clustered between 6,370 and 7,127 MB/s. Against its own CD9P-R sibling at 6,912.4 MB/s, the enterprise drive had a 25% advantage, a meaningful gap given that both use the same BiCS8 flash. 128K Sequential Read (IODepth 64 / NumJobs 1) The read side is the one result in our dataset that’s far from the drive’s rating. At IODepth 64 on a single worker, the CM9-R delivered 9,974.6 MB/s at 801.7 µs, last in the group and well short of both the 14,800 MB/s spec and the 14,235.9 MB/s its CD9P-R sibling posted in the identical configuration. This is a single-job test, and the sweeps below show the CM9-R reading at full pace when work is spread across jobs. However, buyers with single-stream large-block read patterns should note this behavior and test their own pipeline. 64K Random Write The 64K random write sweep placed the CM9-R second in the group, peaking at 9,635.3 MB/s (IODepth 8 / NumJobs 2, at just 103.4 µs), behind only the Micron 9550 MAX at 10,878.1 MB/s. The low-queue-depth entry was the best in the field outright: 3,334.8 MB/s at 18.4 µs at IODepth 1 / NumJobs 1, ahead of the Solidigm PS1030 (2,901.7 MB/s at 21.1 µs) and everything else. The latency ceiling remained controlled, topping out at 1,942.5 µs across the sweep, while several competitors exceeded 2,300 µs and the PS1010 spiked to nearly 6,000 µs. 64K Random Read At 64K reads, the CM9-R posted the best single-stream opening in the group, 1,359.0 MB/s at 45.6 µs at IODepth 1 / NumJobs 1, edging out even the CD9P-R (1,334.0 MB/s), with the rest of the field starting at roughly half that. Just as striking is how little concurrency it needed: the drive reached its 13,402.4 MB/s peak at IODepth 4 / NumJobs 8 with only 148.7 µs of latency, while most of the field needed IODepth 32 configurations to reach theirs. The deep-queue crown went to the Solidigm PS1030 at 14,162.9 MB/s, but nothing in the group serves 64K reads at low concurrency as well as the two KIOXIA drives. 16K Sequential Write One programming note: our 16K coverage is now sequential rather than random, a workload we first broke out in our Micron 9550 MAX review and are carrying across the group, since mid-size sequential streams dominate real deployments, particularly AI pipelines that move ordered data in runs rather than scattering random hits. The change suits the CM9-R. Its 10,812.2 MB/s peak was second only to the Micron 9550 MAX at 10,970.8 MB/s, but where the 9550 MAX needed IODepth 32 / NumJobs 4 and 181.9 µs to reach its number, the CM9-R hit its peak at IODepth 1 / NumJobs 16 with 22.8 µs of latency, the same low-queue-depth signature that defines this drive everywhere else. Single-worker latency of 10.4 µs was again the best in the group, and the drive cleared 10,400 MB/s at three separate points spanning IODepth 1 to 32. 16K Sequential Read On reads, the CM9-R posted 13,393.2 MB/s at IODepth 32 / NumJobs 8, second to its CD9P-R sibling at 13,819.7 MB/s, and it delivered effectively the same figure, 13,390.1 MB/s, at IODepth 16 / NumJobs 8 with half the latency, 149.0 versus 298.3 µs. One honest wrinkle: unlike the random tests, single-stream 16K sequential reads favored the Sandisk SN861 (12.5 µs) and the Micron (13.9 to 21.3 µs) over the KIOXIA pair, which were near 32 µs. Once parallelism was introduced, the CM9-R scaled well, achieving 12,400 MB/s at IODepth 4 / NumJobs 16. 4K Random Write The CM9-R reached 1,502.9K IOPS at IODepth 8 / NumJobs 16 with just 84.6 µs of latency, fourth in the group behind the Micron 7600 MAX (1,781.2K), Solidigm PS1030 (1,595.8K), and 9550 MAX (1,544.2K), and nearly 2.8x its 540K rated figure, which KIOXIA specs at a fixed queue depth. At queue depth 1, it led the entire field with 115.8K IOPS and 8.2 µs, the fastest small-block write response we measured among drives in this group. 4K Random Read The deep-queue 4K read sweep is the CM9-R’s weakest chart: 1,973.4K IOPS at its IODepth 16 / NumJobs 16 peak, seventh of eight, ahead of only the Micron 7600 MAX, and well below the Sandisk SN861’s group-leading 2,555.6K. Note the rating context: KIOXIA’s 3,400K IOPS spec is measured at queue depths beyond our sweep’s IODepth 32 / NumJobs 16 ceiling, so we treat the gap as a difference in test configuration rather than a shortfall, but drives like the SN861 and PS1030 found substantially more within the same sweep. The other half of the chart belongs to the CM9-R: 33.6K IOPS at 29.3 µs at queue depth 1, the lowest 4K read latency in the group, fractionally ahead of the CD9P-R, which made the same trait its signature in June. GDSIO Performance GPU Direct Storage reads are where the CM9-R’s thread scaling shows the same character as its CD9P-R sibling demonstrated in June. The two KIOXIA drives led the 16K block-size segment throughout, and the CM9-R’s peak 16K result of 136.4K IOPS was the best in the entire group, ahead of the CD9P-R’s 134.2K and far clear of the Solidigm pair at the bottom (the PS1030 managed 101.2K). In the 1M segment, the CM9-R peaked at 6.06 GiB/s, second only to the CD9P-R’s 6.16 GiB/s, with the Solidigm PS1010 and Micron 9550 MAX at 6.05 GiB/s behind it. Single-thread 16K read latency of 44.5 µs sat mid-group, effectively identical to the CD9P-R; the Sandisk SN861 held the single-thread edge at 26.1 µs. The write sweep is where the CM9-R quietly sets itself apart. Its 5.51 GiB/s peak (1M block, 16 threads) was second to the Micron 9550 MAX’s 5.69 GiB/s on paper, but the shape of the two lines is the real story: the 9550 MAX dipped to roughly 2.2 GiB/s at 1M/64 in its documented volatility window, while the CM9-R held the most stable line in the group across the entire 1M segment. It also avoided the 128K high-thread collapse exhibited by both Solidigm drives. Single-thread 16K write latency of 21.4 µs tied the CD9P-R for the group’s best. For sustained GPU-adjacent write streams, this was the most predictable drive on the bench. DLIO Checkpoint Performance Our DLIO checkpoint test writes recurring training checkpoints across three passes, and it is a test of consistency as much as speed; lower times are better, and the gap between pass one and pass three shows how a drive behaves once its FTL is working under sustained pressure. One note on this data: checkpoint counts scale with drive capacity, so larger drives, such as the 15.36TB CM9-R, log more checkpoints per run, and per-checkpoint results are not aligned point-for-point across drives. That is why the table below presents pass averages, which normalize each drive’s run into a directly comparable figure; lower is better. On the first pass, the entire comparison group falls within a tight 459- 465-second range, with the CM9-R at 462.8 seconds. The separation comes later. By pass three, the CM9-R settles at 580.9 seconds, mid-pack toward the rear of the group: quicker than the Solidigm PS1030’s 599.2 seconds, but behind the Sandisk SN861 (553.3), Micron 9550 MAX (555.3), and its own CD9P-R sibling (570.6). The pass-three position deserves interpretation rather than alarm. A 1 DWPD read-intensive drive is not bought for sustained checkpoint writes, and the FIO results carry the drive’s actual argument: the group’s best low-queue-depth latency on both reads and writes, delivered consistently. Notably, the CM9-R degraded gradually and predictably across its passes rather than swinging, showing the same steadiness as its GDS write line, which is what checkpoint scheduling cares about once a drive is in the rotation. Buyers who need the checkpoint burst should look at the Micron 9550 family or wait for the CM9-V mixed-use sibling. Conclusion The KIOXIA CM9-R 15.36TB is the first enterprise-class SSD with BiCS8, tested at the capacity point where the series delivers its best write performance, and our data gives it clear framing: the lowest low-queue-depth latency in the comparison group across every workload we ran. At queue depth 1, it led the field outright in 4K reads (33.6K IOPS at 29.3 µs), 4K writes (115.8K IOPS at 8.2 µs), 16K sequential writes (10.4 µs single-worker), and 64K reads (1,359.0 MB/s at 45.6 µs), and it reached its 64K read peak at a fraction of the concurrency its rivals required. In GPU Direct Storage, it produced the group’s best small-block read IOPS at 136.4K and the most stable sustained write line on the bench, peaking at 5.51 GiB/s, without the high-thread volatility seen in several competitors. The deep-queue 4K read peak of 1,973.4K IOPS sat seventh of eight in our sweep (KIOXIA’s 3,400K rating is measured at queue depths beyond our test ceiling, but rivals found more inside the same sweep), and the single-worker 128K sequential read of 9,974.6 MB/s was the one result off from its rating. For latency-sensitive read tiers, OLTP frontends, virtualized estates on dual-port HA infrastructure, and read-heavy AI serving where response time at real-world queue depths matters more than synthetic deep-queue peaks, the CM9-R is the strongest argument yet that BiCS8’s CBA design changes the class. Sustained heavy-write pipelines should look to the CM9-V or the mixed-use drives in this group. Everyone else gets the drive that answers fastest when it matters, in a capacity stack that runs to 30.72TB in E3.S and 61.44TB in 2.5-inch. The CM9-R’s queue-depth-1 sweep also earned it a leaderboard spot: it now holds Best for Latency-Critical Workloads on our Best Enterprise SSDs page.