# Do I need to switch to SGlang/vLLM for Qwen3.8-Flash-Next? — retained record

> Retained reddit record from reddit:LocalLLaMA.

Canonical: https://highsignal.app/data/records/7717b5ecc9e185d2

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:LocalLLaMA)  published 2026-09-17 13:34 ingested 2026-09-18 02:42

[original source ↗](https://www.reddit.com/r/LocalLLaMA/comments/1wiu615/do_i_need_to_switch_to_sglangvllm_for/)

## retained text



```
Setup: 2x5080 + 128gb DDR5

I thought I was cool because I do everything on llama.cpp, but I'm starting to think maybe I should explore vLLM/Sglang seeing how there are not many NVFP4/FP8 .gguf files, and maybe I could be getting better performances out of this model given my setup?

I'm currently getting around 100tps (up to 150-170 if measured as instantaneous rates at different depths) prefill and \~18-20tps decode on Q5\_K\_XL using MTP and ncmoe42 on llama.cpp:

\[Qwen3.8-Flash-Next-UD-Q5\_K\_XL-dual-mtp-ncmoe42-vision\]

alias = Qwen3.8-Flash-Next-Q5-dual-mtp-vision

model = models/mymodels/unsloth/Qwen3.8-Flash-Next-UD-Q5\_K\_XL/Qwen3.8-Flash-Next-UD-Q5\_K\_XL-00001-of-00006.gguf

load-mode = mmap

mlock = false

no-mmap = false

lazy-mode = on

fit = off

ngl = 999

n-cpu-moe = 42

override-tensor = per\_layer\_token\_embd=CPU,blk\\.(42|43|44|45)\\.ffn\_(up|down|gate)\_exps=CUDA0,blk\\.(46|47)\\.ffn\_(up|down|gate)\_exps=CUDA1

device = CUDA0,CUDA1

tensor-split = 1,1

ctx-size = 262144

batch-size = 2048

ubatch-size = 512

cache-type-k = q8\_0

cache-type-v = q8\_0

flash-attn = on

threads = 8

parallel = 1

mmproj = models/mymodels/unsloth/mmproj-Qwen3.8-Flash-Next-BF16.gguf

spec-type = draft-mtp

spec-draft-model = models/mymodels/unsloth/Qwen3.8-Flash-Next-MTP/mtp-Qwen3.8-Flash-Next-shared-Q4\_K\_M.gguf

spec-draft-n-max = 2

jinja = true

reasoning = on

chat-template-kwargs = {"enable\_thinking": true, "preserve\_thinking": true, "reasoning\_effort": "medium"}

temp = 1.0

top-k = 20

top-p = 0.95

min-p = 0.0

presence-penalty = 0.0

repeat-penalty = 1.0
```



document fetched 2026-09-18 00:43 · canonical https://reddit.com/r/LocalLLaMA/comments/1wiu615/do_i_need_to_switch_to_sglangvllm_for
