Memory Bandwidth problems with Intel Sapphire Rapids
retained text
<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1vjghl9/memory_bandwidth_problems_with_intel_sapphire/"> <img alt="Memory Bandwidth problems with Intel Sapphire Rapids" src="https://preview.redd.it/e2he0nek3aih1.png?width=640&crop=smart&auto=webp&s=01fd134a3d40f08fc8d3ef7a2b4630faea185082" title="Memory Bandwidth problems with Intel Sapphire Rapids" /> </a> </td><td> <!-- SC_OFF --><div class="md"><p>I have a Xeon w7-3465 and 4 sticks of RDIMM DDR5-4800 with a theoretical max bandwidth of 153GB/s. I am trying to run DeepSeek-V4-Flash-0731 as it is an MoE and the weights are in MXFP4, so I should roughly process 150GB/6.5GB tokens per second as it has 13B Experts. The problem is, I've only seen around 36-40GB/s max so far, and I tried GPT to solve this and in one iteration he said he got 90GB/s, and said it was about CPU --threads and --threads batch settings mismatch, since I only have 28 physical cores and 56 logical cores. It then tried 28/28 and told me that the inference speed improved by 2.4x. But the speed is still crawling at around 3-4 t/s. Why am I not able to get to the speeds this machine is able to give? The RAM channels are 1-5-8-11 and I've seen around 90GB/s in the watchdog, though this was a one-time event, so it should be configured as 4 channel.</p> </div><!-- SC_ON -->   submitted by   <a href="https://www.reddit.com/user/GetOutOfMyFeedNow"> /u/GetOutOfMyFeedNow </a> <br /> <span><a href="https://i.redd.it/e2he0nek3aih1.png">[link]</a></span>   <span><a href="https://www.reddit.com/r/LocalLLaMA/comments/1vjghl9/memory_bandwidth_problems_with_intel_sapphire/">[comments]</a></span> </td></tr></table>
document fetched 2026-08-09 07:00 · canonical https://reddit.com/r/LocalLLaMA/comments/1vjghl9/memory_bandwidth_problems_with_intel_sapphire