Finally got Qwen 3.8 Next running on my v100 6gpu setup (TP2 PP3)
retained text
Hey everyone,
After wrestling with hardware and engine issues for days, I finally got **Qwen 3.8 Next** running properly on my multi-GPU rig. Thought I’d share the setup journey, benchmarks, and thermal results for anyone trying something similar.
Seeing all the ongoing memes on Reddit about multi-GPU setups turning into absolute space heaters and catching fire, I decided to run some rigorous thermal tests to see for myself.
# he Troubleshooting Odyssey
1. **PCIe Link Speed Issue:** Right after installation, one of the cards dropped to PCIe Gen 1 x16. Spent about 8 hours over two days diagnosing and fixing it.
2. **Finding the Right Engine:**
* Started with `sglang-v100`, but kept hitting continuous OOM crashes.
* Someone on Reddit previously suggested the `pxa` engine, but that threw errors as well.
* Eventually tried `1cat-vllm`, spent some time tweaking it, and finally hit a stable run!
3. **Configuration:**
* Running with **TP2 PP3**.
* Currently, speculative decoding is limited to `speculative=1`. Setting it to 2 throws an OOM due to memory constraints (might look into optimizing this later, but for now, it works).
# Context & Memory Stats
Plaintext
INFO: Available KV cache memory: 8.78 GiB
INFO: GPU KV cache size: 531,288 tokens
INFO: Maximum concurrency for 262,144 tokens per request: 2.03x
# Performance Benchmarks
# 1. Prompt Processing (Prefill)
|**Input Length (Tokens)**|**Speed (tok/s)**|
|:-|:-|
|**1,024 (1K)**|1,389|
|**2,048 (2K)**|2,536|
|**4,096 (4K)**|3,210|
|**8,192 (8K)**|4,336|
|**16,384 (16K)**|4,679|
|**32,768 (32K)**|4,470|
|**65,536 (64K)**|3,820|
|**131,072 (131K)**|2,759|
# 2. Text Generation (MTP Comparison)
|**Output Length (Tokens)**|**Base Speed (No MTP, tok/s)**|**Optimized Speed (MTP Enabled, tok/s)**|
|:-|:-|:-|
|**128**|22.23|41.34|
|**256**|22.32|42.48|
|**512**|22.70|43.04|
|**1024**|22.86|43.28|
|**2048**|22.83|43.38|
|**Average**|**22.59**|**42.70**|
*MTP nearly doubles generation throughput across the board.*
# Thermals & Acoustics
People often meme about multi-GPU rigs turning into space heaters or jet engines, so I ran a thorough thermal/stress test:
* **Stress Test:** Ran `gpu-burn` continuously for 20 minutes.
* **Thermal Equilibrium:** Temperatures peaked at 65°C and stabilized right around **64°C**.
* **Fan Curve:** Based on my fan control script, the fans were only running at around **76%** at 64°C. The cards stay well under 65°C without even needing full blast.
Pretty happy with how stable, cool, and quiet this system turned out.document fetched 2026-09-21 00:37 · canonical https://reddit.com/r/LocalLLaMA/comments/1wl680s/finally_got_qwen_38_next_running_on_my_v100_6gpu