# Qwen3.8-27B and Flash-Next on consumer hardware: engine numbers I could verify, and what they cost with 2026 RAM prices — retained record

> Retained reddit record from reddit:LocalLLaMA.

Canonical: https://highsignal.app/data/records/236a1eb5f6af8103

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:LocalLLaMA)  published 2026-10-19 11:09 ingested 2026-10-20 02:39

[original source ↗](https://www.reddit.com/r/LocalLLaMA/comments/1x9tzf1/qwen3827b_and_flashnext_on_consumer_hardware/)

## retained text



```
I've been keeping a list of the engines that run Qwen3.8-27B and Flash-Next on consumer hardware, mostly to see where my own engine ([capy-flash](https://github.com/ValerioDolci/capy-flash)) stands. Sharing it because most numbers going around aren't comparable. Our rows are measured on my box: 2× RTX 5070 Ti, 96 GB DDR5.

"Verified" means a verified [LocalMaxxing](https://www.localmaxxing.com/en/models) run or a paired run on the same machine; everything else is what the author reports. Decode is tok/s, one request.

# What actually makes the difference

* **Tensor parallel vs pipeline.** TP (capy-flash, SGLang, vLLM, ExLlamaV3, Paiton on AMD) adds up the bandwidth of two cards for one request. Layer split (Strata, llama.cpp) adds memory, not speed.
* **Experts out of VRAM** (Flash-Next). Hot experts in VRAM, misses computed on CPU (Strata, capy-flash, Infernix, fngine). What counts is how many experts miss per step.
* **Quant format.** NVFP4 is better quality but more bytes. 2–3-bit IQ GGUF is faster at about twice the KL (Strata's numbers: 0.144 vs 0.070).
* **Drafters.** MTP gives 1.3–1.7×. On the 27B, DFlash2 and DSpark give 2–3×: that's where most 400+ tok/s headlines come from. A DFlash2 drafter for Flash-Next came out today ([incoai](https://huggingface.co/incoai/Qwen3.8-Flash-Next-DFlash2)), so far only run on SGLang.
* **Single-model kernels** (MegaCapybara, NInfer, capy-flash) beat general engines by 2–6× at batch 1. vLLM and SGLang win once you have many users.

# Qwen3.8-27B

|Hardware|Engine|tok/s|Verified|
|:-|:-|:-|:-|
|1× 5090|MegaCapybara|313–540 (greedy)|no|
|1× 5090|SparkInfer (DSpark)|265 avg, 420 code|no|
|2× 5070 Ti|capy-flash (DFlash2)|379|yes|
|7900 XTX / R9700|hipfire / Lucebox|318 / 208–228|no|
|2× 5090|SGLang fork (TP2)|315|yes|
|1× 5090|TensorFold|220 chat, 305 code|no|
|1× 3090|tqLLM / ExLlamaV3|273 / 242|yes|
|1× 5090|llama.cpp (DFlash2)|205|yes|
|1× 5060 Ti 16 GB|tqLLM (3-bit)|156|yes|
|2× 5060 Ti|lynx-gt (TP2)|126|no|
|Mac M5 Pro|Splash|74|no|

Prefill: NInfer 12.8k tok/s on a 5090 (reported), capy-flash 9.2k at 4–16k on 2× 5070 Ti without P2P (boards with P2P get \~6k for now), MegaCapybara \~7.6k.

# Qwen3.8-Flash-Next (125B MoE, 6B active)

|Hardware|Engine|tok/s|Verified|
|:-|:-|:-|:-|
|5090 + 5060 Ti + 128 GB|Basalt (NVFP4)|239–479|no|
|2× R9700|Paiton (TP2, W3A8 / W4A8)|272 / 240|no|
|1× 5090|fngine / Infernix|146–209 / 140|no|
|2× 5070 Ti + 96 GB|Strata IQ3\_S|195|yes|
|2× 5070 Ti + 96 GB|capy-flash (NVFP4)|178|yes|
|1× 5090 + 128 GB|strata-nvfp4 fork|177|no|
|2× 5090|llama.cpp fork|142|yes|
|Mac M5 Max 128 GB|MTPLX / Strata-Mac|126 / 119|no|
|2× 5070 Ti + 96 GB|Strata NVFP4|120|yes|
|2× 3090 + 128 GB|vLLM + CPU experts|101–111|no|
|5060 Ti + 5080|Strata IQ4\_XS|108|yes|
|DGX Spark|ninfer-gb10|60|no|
|1× 5060 Ti 16 GB|Strata IQ3\_S / llama.cpp|48 / 30–40|no|

Short-prompt decode isn't the whole story. I replayed a real 53-turn coding session (growing to 81k tokens) on my box: capy-flash 177 s, Strata IQ3\_S 197 s, Strata NVFP4 268 s. Prefill at 4k / 16k / 64k: capy-flash 2.4k / 4.6k / 4.8k, Strata IQ3\_S 2.2k / 3.8k / 5.2k.

# What it costs

EU street prices this month, new, VAT included: RTX 5090 \~€9.5k, R9700 €2.2k, 5070 Ti €1.3k, 5060 Ti 16 GB €740, **96 GB DDR5 €1.8k**, 128 GB \~€2.4k. RAM is counted where the model needs it; CPU, board and PSU are not.

**Flash-Next**

|Setup|Cost|Engine|tok/s|tok/s per €1k|
|:-|:-|:-|:-|:-|
|2× R9700, model in VRAM|€4.4k|Paiton|272|62 (reported)|
|2× 5070 Ti + 96 GB|€4.4k|Strata IQ3\_S|195|44|
|2× 5070 Ti + 96 GB|€4.4k|capy-flash|178|40|
|MacBook Pro M5 Max 128 GB|\~€5.7k|MTPLX|126|22 (reported)|
|1× 5090 + 128 GB|€11.9k|strata-nvfp4 fork|177|15 (reported)|
|DGX Spark|\~€6.3k|ninfer-gb10|60|10 (reported)|
|2× 5090|€19k|llama.cpp fork|142|7|

**27B** (fits in VRAM)

|Setup|Cost|Engine|tok/s|tok/s per €1k|
|:-|:-|:-|:-|:-|
|1× 5060 Ti|€740|tqLLM 3-bit|156|211|
|2× 5070 Ti|€2.6k|capy-flash|379|146|
|2× 5060 Ti|€1.5k|lynx-gt|126|85 (reported)|
|1× 5090|€9.5k|MegaCapybara / llama.cpp|440 / 205|46 / 22|
|2× 5090|€19k|SGLang fork|315|17|

At these prices two mid-range cards beat a 5090 by 3–10× per euro. The surprise for me was RAM: 96 GB of DDR5 now costs more than one of my GPUs, so on an experts-in-RAM box it's about 40 % of the bill. That puts 2× R9700 with the whole model in VRAM on top for Flash-Next, if Paiton's numbers hold up, and makes 64 GB builds worth chasing.

Nobody seems to run Flash-Next on 2× 5060 Ti yet. If you have verified runs I missed, post them and I'll add them.
```



document fetched 2026-10-20 00:42 · canonical https://reddit.com/r/LocalLLaMA/comments/1x9tzf1/qwen3827b_and_flashnext_on_consumer_hardware
