← sources

Llama.cpp setup Qwen-flash-next Vulkan Q4 with Expert cache.

source: reddit (reddit:LocalLLaMA)published 2026-11-01 01:39ingested 2026-11-02 02:42

retained text

I just setup my server:

64GB DDR4 2133 4CH (Broadwell Xeon)  
2 x RX 6800 16GB  
1 x Rx 580 2048sp (16GB)

I am using the Q4 of Qwen Flash Next, with the prompt:  
sudo ./llama-server -m ./models/Qwnflash180b-Q4.gguf --host [0.0.0.0](http://0.0.0.0) \--port 8081 -ngl 99 -fa on  --moe-cache-mib 16384 --log-verbosity 4 --metrics -mg 1

I am able to get 19-21t/s generation at full context (~~262K~~  64K) (which is why i capped the moe-cache-mib at 16GB)

**However note: when I set any context size with the -c or --ctx-size flags,** **~~performance tanked to 8t/s even when the context was smaller (100K).~~** **it seems you need to let llama.cpp allocate the context itself.** ***The context was automatically 64k, I was actually increasing it, causing layers to spill over to system ram.***

PCIe slots:

gpu0 rx6800 x16 3.0  
gpu1 rx6800 x16 3.0  
gpu2 rx580   x4 3.0

document fetched 2026-11-02 00:37 · canonical https://reddit.com/r/LocalLLaMA/comments/1xkr0e6/llamacpp_setup_qwenflashnext_vulkan_q4_with