# tandem.cpp: Qwen3.8-Flash-Next at 97 tok/s on a single 4090 + 64 GB RAM, 3.3× llama.cpp, works for any MoE — retained record

> Retained reddit record from reddit:LocalLLaMA.

Canonical: https://highsignal.app/data/records/a1ef22bbee904d3a

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:LocalLLaMA)  published 2026-11-01 20:23 ingested 2026-11-02 02:41

[original source ↗](https://www.reddit.com/r/LocalLLaMA/comments/1xldent/tandemcpp_qwen38flashnext_at_97_toks_on_a_single/)

## retained text



```
tandem.cpp is a patch set for llama.cpp that makes MoE models that don't fit in VRAM run faster. It keeps the most-used experts on the GPU and lets the CPU help with the rest, for any MoE model llama.cpp supports. Single-user decode is the target.

Consumer PC (i5-12400F, 64 GB DDR4, RTX 4090), GSM8K, decode tok/s:

|Model|llama.cpp|tandem.cpp|Strata|
|:-|:-|:-|:-|
|Qwen3.8-Flash-Next IQ3\_S + MTP, greedy|29.8|**97.3** (3.3×)|103.6|
|same, Qwen's default sampling|29.4|**96.1** (3.3×)|98.1|
|gpt-oss-120b, greedy|27.8|**51.5** (1.85×)|—|
|gpt-oss-120b, recommended sampling|27.7|**48.0** (1.73×)|—|

Tuned for Qwen3.8 only, but other MoE models that don't fit in VRAM gain too: typically 1.4–2.1× (gpt-oss-120b, GLM-4.5-Air, Qwen3.6-35B-A3B, DeepSeek-V4.1). Prompt processing is 1.4–1.8× faster on a 64k prompt.

Strata is shown in its exact mode (every routed expert computed, like llama.cpp). Its default skips low-ranked experts, which is faster but approximate (127.1 here).

Independent project, not affiliated with ggml-org. Build, run commands, and full results on 4 machines: [https://github.com/ewhacc/tandem.cpp](https://github.com/ewhacc/tandem.cpp)
```



document fetched 2026-11-02 00:36 · canonical https://reddit.com/r/LocalLLaMA/comments/1xldent/tandemcpp_qwen38flashnext_at_97_toks_on_a_single
