# Qwen3.8-Flash-Next on 2x RTX 5090 with 504 tok/s decode: QEngine 1.3.1 decodes ~1.5-1.9x faster than the next-best engine I've tried (numbers, caveats, and what still bites) — retained record

> Retained reddit record from reddit:LocalLLaMA.

Canonical: https://highsignal.app/data/records/978e202f6e9b9b3a

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:LocalLLaMA)  published 2026-10-31 01:09 ingested 2026-11-01 02:40

[original source ↗](https://www.reddit.com/r/LocalLLaMA/comments/1xjx0ez/qwen38flashnext_on_2x_rtx_5090_with_504_toks/)

## retained text



```
https://github.com/bpx03/QEngine
(not my repo, credit goes to bpx03)

QEngine was built for 3x RTX 3090s. I was able to compile for RTX 5090s.

I've now run Qwen3.8-Flash-Next on the same box through four different engines. QEngine (bpx03, v1.3.1) is the fastest for single-stream decode by a clear margin, level with Strata on agentic prompt latency, and it has a KV-in-RAM mode that makes returning to idle conversations much cheaper. Details below, including the parts that make it look less good.

**Rig:** 2x RTX 5090 (32 GB each), Threadripper PRO 3975WX, 251 GB RAM, 262K context. QEngine's INT4 AutoRound build of Flash-Next. The README only lists sm_86, so I built it for sm_120 by changing `CMAKE_CUDA_ARCHITECTURES` from 86 to 120 (nvcc 13.3, g++-13). No other changes. It's up and serving 53 s after launch.

## Speed (club-3090 `bench.sh`, same script and prompts for every engine)


| Engine / quant | Decode narrative / code (tok/s) | Prefill @10K / @90K (tok/s) |
|---|---|---|
| **QEngine 1.3.1** (KV in VRAM, default) | **370 / 439** | ~5.7K / ~7.3K |
| **QEngine 1.3.1** (`FLASH_KV=ram`) | **406 / 504** | ~6.2K @10K |
| Strata ISTA IQ3_S | 214 / 293 | ~5.1K / ~10.0K |
| Strata UD-IQ4_XS | 180 / 239 | 4,264 / 8,399 |
| bucko vLLM `fast-256k` (tuned) | 135 / 141 | 2,888 / 3,886 |
| bucko vLLM ple (shipped config) | 120 / 118 | ~2.25K / ~3.1K |
| Infernix (sdekok fork, TP2) | 14-18 | n/a |

**Read the decode column with care.** QEngine's expert cache adapts to recent use, and `bench.sh` repeats the same prompts, so these are close to best case. In an earlier test on varied traffic I measured about 300-400 tok/s single-stream (hermes agent traffic ran 330-470 per request). Strata's cache holds ~99.5% of routed expert mass, so repeating a prompt doesn't help it. The fair comparison is roughly **1.4-1.9x Strata and ~3x bucko**.

Prefill is a split result. QEngine is faster at 10K and slower than Strata at 90K, because Strata's rate climbs with depth and QEngine's stays flat.

## Agentic latency (12-turn tool-use bench, context grows to 35K)

Turn-12 prompt read, with the prefix cache working:

- QEngine 1.3.1: **1.75 s**
- Strata IQ4_XS: 1.7-1.9 s
- ThinkingCap 27B dual-fast: 2.07 s
- exl3: 3.5-3.9 s
- bucko: 4.36 s

QEngine and Strata are level here (QEngine's rc4 build read 1.4 s in the same bench, so this one is within noise of "about 1.5-1.8 s"). Both are about 2x faster than bucko.

## Idle conversations: `FLASH_KV=ram`

1.3.1 can keep the KV cache in pinned host RAM (13.5 GB pool, about 1M tokens) instead of VRAM. I tested 8 conversations of ~58K tokens each, then 400K tokens of other traffic, then went back to each one:

- **KV in RAM:** 8 of 8 came back warm, 1.56-2.01 s to first reply.
- **KV in VRAM:** 2 of 8 warm. The other six re-prefilled in 8.7-9.4 s, about the same as a cold start (8.85 s). Answers were still correct, so you lose latency, not correctness.

The RAM mode also decoded ~10% faster single-stream here, but ~10% slower with 8 concurrent 64K streams (124 vs 139 tok/s per stream). One boot per mode, one pass.

## Quality and stability

Through the omp agent harness (44-task assistant set, effort low, one draw): **44/44 on 1.3.1**, against 42/44 twice on 1.3. At medium and xhigh on 1.3: 43/44 each. A 6-task sub-agent delegation set passed 6/6 with and without delegation. The 1.3 to 1.3.1 gap is one task, so treat it as noise.

Coding check (aider-polyglot-30, thinking on, effort low, single pass), **on the rc4 build**: QEngine 26/30. Strata UD-IQ4_XS 24/30, UD-Q4_K_XL 25/30 (24-26 across repeats), ThinkingCap 27B 17/30. 26 vs 24-26 is inside one Strata config's repeat spread, so the honest claim is that the INT4 AutoRound quant is not worse at coding, not that it's better. I haven't re-run this on 1.3.1.

I also ran my usual multi-pack agent/tool-use suite earlier; QEngine landed level with the other Flash-Next engines, within noise.

## Bottom line

On 2x5090, QEngine 1.3.1 is the fastest Flash-Next engine I've measured for single-stream decode, level with Strata on agentic prompt latency, and the KV-in-RAM mode is a real win for long idle sessions. Coding quality looks level with Strata's GGUF quants. The costs are tight VRAM, a build-from-source step, and less runtime history than vLLM or Strata.

##

Please try, the dev is very encouraging. Additional resources available at https://github.com/noonghunna/club-3090. The Club also welcomes participation!
```



document fetched 2026-11-01 00:38 · canonical https://reddit.com/r/LocalLLaMA/comments/1xjx0ez/qwen38flashnext_on_2x_rtx_5090_with_504_toks
