Qwen 3.8 on 3090 comparisons
retained text
I’m getting \~35 tok/s with Qwen3.8-27B on my RTX 3090. Another setup gets \~65 tok/s on the same GPU. Is mine slow? Mine: • Q5 weights • Q8 KV cache • 32K context • 4 parallel slots • MTP off The faster setup: • Q4 weights • Q4 KV cache • 131K context • 1 slot • MTP on The trade-offs: Q5 vs Q4: Different quantization precision Q8 vs Q4 KV: cache precision versus context capacity 4 slots vs 1: concurrency versus single-user speed MTP (Multi-token prediction) on vs off: speculative speed versus memory and verification work Is there anything I'm missing? Any other variables that I should consider? For context, I'm using Llama.cpp
document fetched 2026-09-01 00:41 · canonical https://reddit.com/r/LocalLLaMA/comments/1w3lg41/qwen_38_on_3090_comparisons