Second-gen Optane PMem server running Kimi K3 (2.8T params) on CPU only, plus a critical thermal warning that could save your RAM
retained text
Hey everyone! Some of you may remember u/APFrisco's post about running a 1T-parameter model (Kimi K2.5) at around 4 tok/s on a first-gen Optane server with 768GB of PMem 100 series in Memory Mode, it was even covered by Tom's Hardware. That post was a big inspiration for me, and I've now built a second-generation version using Optane Persistent Memory 200 series ("Barlow Pass") on a 3rd-gen Xeon Scalable platform with 40 core CPU.
The most important part of this post: Optane DIMMs run HOT, and the heat will kill your DRAM
This is the thing almost nobody talks about, and it's an extremely relevant fact for anyone considering a build like this: under sustained LLM inference (constant heavy reads across every module), Optane DCPMMs get extremely hot. This isn't speculation. Intel rates these modules at 15 to 18W TDP each, throttling kicks in at 83C and thermal shutdown at 85C, and Intel's thermal specs assume datacenter forced airflow (multiple m/s across the DIMMs) that simply does not exist in a workstation or desktop chassis.
Here's the real danger: without powerful directed airflow, the heat coming off the Optane sticks will quickly burn out the more temperature-sensitive DRAM modules sitting alongside them, and replacement ECC DRAM can be very expensive, easily more than you paid for the used Optane itself. The Optane can take the heat better than your RAM can.
My solution: 2x 12W blower fans aimed directly at the memory modules. It's loud, but temps are now well under control even during long generation runs. PHOTO: blower fan setup blowing across the Optane DIMMs
If you're building anything with Optane PMem, first or second gen, treat active DIMM cooling as mandatory, not optional. Your RAM will thank you.
Preliminary results (CPU-only, llama.cpp, App Direct mode)
Unlike APFrisco's build, I'm running the Optane in App Direct mode (model mmap'd from a DAX-mounted fsdax namespace) with pure CPU inference, no GPU at all:
Model (Kimi K3, 2.8T total / 104B active),Tokens/sec
1-bit quant: 0.85 tok/s
3-bit quant: 0.75 tok/s
Yes, it's slow, but this is a 2.8-trillion-parameter frontier-class model running entirely from memory that cost a fraction of equivalent DRAM, on a CPU-only box.
Caveats and next steps: These are preliminary numbers in App Direct mode, and I haven't tuned thread counts or NUMA affinity yet. From here I plan to add GPUs for acceleration, and to experiment with loading parts of the model into actual DRAM, which is faster than Optane, a hybrid memory-tier approach. More benchmarks to come.
Happy to answer questions about the build, the cooling setup, or App Direct configuration. And seriously, point a fan at your Optane sticks before they take your DRAM out with them.document fetched 2026-11-03 00:37 · canonical https://reddit.com/r/homelab/comments/1xlljut/secondgen_optane_pmem_server_running_kimi_k3_28t