← sources

Finally got Qwen 3.8 Next running on my v100 6gpu setup (TP2 PP3)

source: reddit (reddit:LocalLLaMA)published 2026-09-20 03:44ingested 2026-09-21 02:38

retained text

Hey everyone,

After wrestling with hardware and engine issues for days, I finally got **Qwen 3.8 Next** running properly on my multi-GPU rig. Thought I’d share the setup journey, benchmarks, and thermal results for anyone trying something similar.



Seeing all the ongoing memes on Reddit about multi-GPU setups turning into absolute space heaters and catching fire, I decided to run some rigorous thermal tests to see for myself.

# he Troubleshooting Odyssey

1. **PCIe Link Speed Issue:** Right after installation, one of the cards dropped to PCIe Gen 1 x16. Spent about 8 hours over two days diagnosing and fixing it.
2. **Finding the Right Engine:**
   * Started with `sglang-v100`, but kept hitting continuous OOM crashes.
   * Someone on Reddit previously suggested the `pxa` engine, but that threw errors as well.
   * Eventually tried `1cat-vllm`, spent some time tweaking it, and finally hit a stable run!
3. **Configuration:**

* Running with **TP2 PP3**.
* Currently, speculative decoding is limited to `speculative=1`. Setting it to 2 throws an OOM due to memory constraints (might look into optimizing this later, but for now, it works).

# Context & Memory Stats

Plaintext

    INFO: Available KV cache memory: 8.78 GiB
    INFO: GPU KV cache size: 531,288 tokens
    INFO: Maximum concurrency for 262,144 tokens per request: 2.03x

# Performance Benchmarks

# 1. Prompt Processing (Prefill)

|**Input Length (Tokens)**|**Speed (tok/s)**|
|:-|:-|
|**1,024 (1K)**|1,389|
|**2,048 (2K)**|2,536|
|**4,096 (4K)**|3,210|
|**8,192 (8K)**|4,336|
|**16,384 (16K)**|4,679|
|**32,768 (32K)**|4,470|
|**65,536 (64K)**|3,820|
|**131,072 (131K)**|2,759|

# 2. Text Generation (MTP Comparison)

|**Output Length (Tokens)**|**Base Speed (No MTP, tok/s)**|**Optimized Speed (MTP Enabled, tok/s)**|
|:-|:-|:-|
|**128**|22.23|41.34|
|**256**|22.32|42.48|
|**512**|22.70|43.04|
|**1024**|22.86|43.28|
|**2048**|22.83|43.38|
|**Average**|**22.59**|**42.70**|

*MTP nearly doubles generation throughput across the board.*

# Thermals & Acoustics

People often meme about multi-GPU rigs turning into space heaters or jet engines, so I ran a thorough thermal/stress test:

* **Stress Test:** Ran `gpu-burn` continuously for 20 minutes.
* **Thermal Equilibrium:** Temperatures peaked at 65°C and stabilized right around **64°C**.
* **Fan Curve:** Based on my fan control script, the fans were only running at around **76%** at 64°C. The cards stay well under 65°C without even needing full blast.

Pretty happy with how stable, cool, and quiet this system turned out.

document fetched 2026-09-21 00:37 · canonical https://reddit.com/r/LocalLLaMA/comments/1wl680s/finally_got_qwen_38_next_running_on_my_v100_6gpu