Four engines, seven models, one Radeon AI PRO R9700: local LLM benchmarks
retained text
Ran the same R9700 through four inference engines and seven models. Full write-up with the tables and the prefill numbers: [https://www.damiansromek.pl/en/tech/blog/local-llm-benchmarks-r9700/](https://www.damiansromek.pl/en/tech/blog/local-llm-benchmarks-r9700/) Headline numbers: | Setup | Model | Result | |---|---|---| | Paiton vLLM (3-bit weights, 4-bit KV, speculator) | Qwen3.8 | 173.8 tok/s median single-stream, 684 tok/s aggregate at 8 streams | | Strata | Qwen3.8-27B | 83.8 tok/s | | Strata | DeepSeek-V4.1-Flash, full quality | 26-27 tok/s | | RTX 5090, llama.cpp + DFlash2 | reference | 289.2 tok/s mean decode | Same card and same base model family, and Paiton comes out 2.07x faster than Strata. I suspect the serving stack matters more than the GPU here :) Concurrency is where the tokens per watt live: 0.58-0.61 tok/W single-stream, 2.3-2.4 tok/W at 8 streams. The hidden tests were the part that surprised me. 84-94% of hidden tests passed, but only 0-3 of 10 tasks came out fully green. Which of these numbers would you want me to re-run first, or which engine should I add next?
document fetched 2026-11-20 00:42 · canonical https://reddit.com/r/LocalLLaMA/comments/1y0i8ww/four_engines_seven_models_one_radeon_ai_pro_r9700