Qwen3.8-27B and Flash-Next on consumer hardware: engine numbers I could verify, and what they cost with 2026 RAM prices
retained text
I've been keeping a list of the engines that run Qwen3.8-27B and Flash-Next on consumer hardware, mostly to see where my own engine ([capy-flash](https://github.com/ValerioDolci/capy-flash)) stands. Sharing it because most numbers going around aren't comparable. Our rows are measured on my box: 2× RTX 5070 Ti, 96 GB DDR5. "Verified" means a verified [LocalMaxxing](https://www.localmaxxing.com/en/models) run or a paired run on the same machine; everything else is what the author reports. Decode is tok/s, one request. # What actually makes the difference * **Tensor parallel vs pipeline.** TP (capy-flash, SGLang, vLLM, ExLlamaV3, Paiton on AMD) adds up the bandwidth of two cards for one request. Layer split (Strata, llama.cpp) adds memory, not speed. * **Experts out of VRAM** (Flash-Next). Hot experts in VRAM, misses computed on CPU (Strata, capy-flash, Infernix, fngine). What counts is how many experts miss per step. * **Quant format.** NVFP4 is better quality but more bytes. 2–3-bit IQ GGUF is faster at about twice the KL (Strata's numbers: 0.144 vs 0.070). * **Drafters.** MTP gives 1.3–1.7×. On the 27B, DFlash2 and DSpark give 2–3×: that's where most 400+ tok/s headlines come from. A DFlash2 drafter for Flash-Next came out today ([incoai](https://huggingface.co/incoai/Qwen3.8-Flash-Next-DFlash2)), so far only run on SGLang. * **Single-model kernels** (MegaCapybara, NInfer, capy-flash) beat general engines by 2–6× at batch 1. vLLM and SGLang win once you have many users. # Qwen3.8-27B |Hardware|Engine|tok/s|Verified| |:-|:-|:-|:-| |1× 5090|MegaCapybara|313–540 (greedy)|no| |1× 5090|SparkInfer (DSpark)|265 avg, 420 code|no| |2× 5070 Ti|capy-flash (DFlash2)|379|yes| |7900 XTX / R9700|hipfire / Lucebox|318 / 208–228|no| |2× 5090|SGLang fork (TP2)|315|yes| |1× 5090|TensorFold|220 chat, 305 code|no| |1× 3090|tqLLM / ExLlamaV3|273 / 242|yes| |1× 5090|llama.cpp (DFlash2)|205|yes| |1× 5060 Ti 16 GB|tqLLM (3-bit)|156|yes| |2× 5060 Ti|lynx-gt (TP2)|126|no| |Mac M5 Pro|Splash|74|no| Prefill: NInfer 12.8k tok/s on a 5090 (reported), capy-flash 9.2k at 4–16k on 2× 5070 Ti without P2P (boards with P2P get \~6k for now), MegaCapybara \~7.6k. # Qwen3.8-Flash-Next (125B MoE, 6B active) |Hardware|Engine|tok/s|Verified| |:-|:-|:-|:-| |5090 + 5060 Ti + 128 GB|Basalt (NVFP4)|239–479|no| |2× R9700|Paiton (TP2, W3A8 / W4A8)|272 / 240|no| |1× 5090|fngine / Infernix|146–209 / 140|no| |2× 5070 Ti + 96 GB|Strata IQ3\_S|195|yes| |2× 5070 Ti + 96 GB|capy-flash (NVFP4)|178|yes| |1× 5090 + 128 GB|strata-nvfp4 fork|177|no| |2× 5090|llama.cpp fork|142|yes| |Mac M5 Max 128 GB|MTPLX / Strata-Mac|126 / 119|no| |2× 5070 Ti + 96 GB|Strata NVFP4|120|yes| |2× 3090 + 128 GB|vLLM + CPU experts|101–111|no| |5060 Ti + 5080|Strata IQ4\_XS|108|yes| |DGX Spark|ninfer-gb10|60|no| |1× 5060 Ti 16 GB|Strata IQ3\_S / llama.cpp|48 / 30–40|no| Short-prompt decode isn't the whole story. I replayed a real 53-turn coding session (growing to 81k tokens) on my box: capy-flash 177 s, Strata IQ3\_S 197 s, Strata NVFP4 268 s. Prefill at 4k / 16k / 64k: capy-flash 2.4k / 4.6k / 4.8k, Strata IQ3\_S 2.2k / 3.8k / 5.2k. # What it costs EU street prices this month, new, VAT included: RTX 5090 \~€9.5k, R9700 €2.2k, 5070 Ti €1.3k, 5060 Ti 16 GB €740, **96 GB DDR5 €1.8k**, 128 GB \~€2.4k. RAM is counted where the model needs it; CPU, board and PSU are not. **Flash-Next** |Setup|Cost|Engine|tok/s|tok/s per €1k| |:-|:-|:-|:-|:-| |2× R9700, model in VRAM|€4.4k|Paiton|272|62 (reported)| |2× 5070 Ti + 96 GB|€4.4k|Strata IQ3\_S|195|44| |2× 5070 Ti + 96 GB|€4.4k|capy-flash|178|40| |MacBook Pro M5 Max 128 GB|\~€5.7k|MTPLX|126|22 (reported)| |1× 5090 + 128 GB|€11.9k|strata-nvfp4 fork|177|15 (reported)| |DGX Spark|\~€6.3k|ninfer-gb10|60|10 (reported)| |2× 5090|€19k|llama.cpp fork|142|7| **27B** (fits in VRAM) |Setup|Cost|Engine|tok/s|tok/s per €1k| |:-|:-|:-|:-|:-| |1× 5060 Ti|€740|tqLLM 3-bit|156|211| |2× 5070 Ti|€2.6k|capy-flash|379|146| |2× 5060 Ti|€1.5k|lynx-gt|126|85 (reported)| |1× 5090|€9.5k|MegaCapybara / llama.cpp|440 / 205|46 / 22| |2× 5090|€19k|SGLang fork|315|17| At these prices two mid-range cards beat a 5090 by 3–10× per euro. The surprise for me was RAM: 96 GB of DDR5 now costs more than one of my GPUs, so on an experts-in-RAM box it's about 40 % of the bill. That puts 2× R9700 with the whole model in VRAM on top for Flash-Next, if Paiton's numbers hold up, and makes 64 GB builds worth chasing. Nobody seems to run Flash-Next on 2× 5060 Ti yet. If you have verified runs I missed, post them and I'll add them.
document fetched 2026-10-20 00:42 · canonical https://reddit.com/r/LocalLLaMA/comments/1x9tzf1/qwen3827b_and_flashnext_on_consumer_hardware