← sources

Qwen3.8-27B FP8 dual GPUs

source: reddit (reddit:LocalLLaMA)published 2026-09-27 11:15ingested 2026-09-28 02:41

retained text

**Hardware**

* RTX 5090 (32 GB) + RTX 4070 Ti Super (16 GB, PCIe x1) = 48 GB VRAM
* 32 GB DDR5-6200, Arch Linux, KDE on the 5090

**Setup**

Huihui Qwen3.8-27B abliterated INT8 W8A16 + DFlash2 drafter (K=7), vLLM 0.30.0, pipeline parallel:

* 4070 Ti Super: vision encoder, layers 0-20
* 5090: layers 21-63, lm_head, drafter

262K context, FP8 KV, 2 slots.

**Benchmarks** (single request, thinking off, fresh context per depth)

|Depth|Prefill|TTFT|Decode (code)|Decode (prose)|
|:-|:-|:-|:-|:-|
|2k|2,922 t/s|0.7 s|184 t/s|72 t/s|
|32k|3,001 t/s|10.7 s|145 t/s|70 t/s|
|62k|2,698 t/s|23.0 s|152 t/s|71 t/s|
|92k|2,444 t/s|37.7 s|157 t/s|63 t/s|
|122k|2,229 t/s|54.8 s|147 t/s|68 t/s|
|152k|2,051 t/s|74.2 s|133 t/s|67 t/s|
|182k|1,900 t/s|95.9 s|143 t/s|61 t/s|
|212k|1,771 t/s|119.8 s|150 t/s|62 t/s|
|242k|1,658 t/s|146.0 s|138 t/s|60 t/s|
|260k|1,596 t/s|163.0 s|134 t/s|56 t/s|

Code decodes faster because the drafter's guesses are accepted ~70% of the time vs ~22% on prose. Follow-up turns hit the prefix cache (1.3 s TTFT at 260k). Needs patched vLLM, see repo.

The 4070 Ti Super sat collecting dust for two months because I assumed PCIe x1 would kneecap it. Apparently not.

My full setup: [https://github.com/ExTV/dual-gpus-vllm](https://github.com/ExTV/dual-gpus-vllm)

document fetched 2026-09-28 00:39 · canonical https://reddit.com/r/LocalLLaMA/comments/1wrhmdn/qwen3827b_fp8_dual_gpus