← sources

Qwen3.8 27b hitting 6,645 tok/s on prefill and 316 tok/s decode peak on Saber

source: reddit (reddit:LocalLLaMA)published 2026-11-06 17:15ingested 2026-11-07 02:42

retained text

I know what you are thinking "another inference engine?", yeah there has been a flood of them I feel now.  I have been running this for over a month now and I have been on the fence to even release it, but I decided to as I took a different approach that others currently available.

Saber is a inference engine built in Rust targeting AMD R9700 GPU's.  It currently supports TP 1, 2 or 4 and Qwen 3.8 27b and Flash next.  Anyways here are the BetterBench results for Qwen3.8 27b, flash next is very fast also.  Saber beats existing available engines on both 27b and flash next.

[BetterBench Qwen 3.8 27b Unsloth NVFP4 Results](https://preview.redd.it/jqp4n2gwx00i1.png?width=1093&format=png&auto=webp&s=ff1fbff55cc2f1e025d972e6963e67fffeffb975)

[https://github.com/GGZ14/saber-engine](https://github.com/GGZ14/saber-engine)  
[https://x.com/bkuyper](https://x.com/bkuyper)

AMD Ryzen 9900X  
Taichi X870E  
64GB RAM  
2xR9700

document fetched 2026-11-07 00:40 · canonical https://reddit.com/r/LocalLLaMA/comments/1xpicjz/qwen38_27b_hitting_6645_toks_on_prefill_and_316