# Qwen3.8-Flash-Next FP8 running at 524K context on 2x RTX PRO 6000 with vLLM — found an MTP long-context issue — retained record

> Retained reddit record from reddit:LocalLLaMA.

Canonical: https://highsignal.app/data/records/a6a8ddd9c280a844

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:LocalLLaMA)  published 2026-08-28 18:05 ingested 2026-08-29 02:38

[original source ↗](https://www.reddit.com/r/LocalLLaMA/comments/1w0xzxu/qwen38flashnext_fp8_running_at_524k_context_on_2x/)

## retained text



```
Got Qwen3.8-Flash-Next FP8 running at **524K context** on 2x RTX PRO 6000 with vLLM.

Current setup is TP2 + EP2, MTP3, PLE CPU offload, prefix caching off, chunked prefill on, and YaRN 2x.

Main Args:

    --tensor-parallel-size 2
    --enable-expert-parallel
    --gpu-memory-utilization 0.85
    --max-model-len 524288
    --max-num-seqs 32
    --max-num-batched-tokens 16384
    --speculative-config '{"method":"mtp","num_speculative_tokens":3}'
    --no-enable-prefix-caching
    --enable-chunked-prefill
    --no-enable-flashinfer-autotune

Env:

    VLLM_PLE_CPU_OFFLOAD=1
    VLLM_ALLOW_LONG_MAX_MODEL_LEN=1

I hit an interesting issue when extending beyond the native 262K context. The target model correctly moved to 524K, but the Qwen3.8 MTP draft was still being created at 262K. That caused Mamba/cache geometry validation failures when prefix caching was disabled.

I ended up patching the Qwen3.8 MTP config so the draft inherits the target `max_model_len` before vLLM builds its cache config.

Now it boots cleanly with:

    GPU KV cache size: 654,980 tokens
    Maximum concurrency for 524,288 tokens/request: 1.25x

EP is also working at 256/512 experts per GPU, and vLLM automatically selected the DeepGEMM FP8 MoE backend.

One remaining thing I’m testing: MTP3 falls back to rebuilding QSA attention metadata between draft steps, so I want to benchmark MTP1 vs MTP2 vs MTP3 and see which actually gives the best real-world tok/s.

Curious if anyone else running Flash-Next on SM120/vLLM has run into the same long-context MTP draft issue or has a cleaner upstream fix.

\# Update.  
Model stable so far, with about 135tk/s aggregated

https://preview.redd.it/iaqdpo8m27mh1.png?width=1651&format=png&auto=webp&s=3336e38868512ca782c7eb1771d3924fda42c4ae

  
 
```



document fetched 2026-08-29 00:43 · canonical https://reddit.com/r/LocalLLaMA/comments/1w0xzxu/qwen38flashnext_fp8_running_at_524k_context_on_2x
