← sources

RustingStrata running Qwen3.6:35b-a3b with Q5 and MTP vs llama.cpp

source: reddit (reddit:LocalLLaMA)published 2026-11-08 16:48ingested 2026-11-09 02:37

retained text

People asked how RustingStrata does with a bigger quant, so here's Q5\_K\_M on the same 3060 12 GB + Ryzen 5 7600X + 32GB DDR5 RAM.

Model is unsloth's Qwen3.6-35B-A3B UD-Q5\_K\_M (the MTP version), 25 GiB, so about two thirds of the experts live on the CPU. llama.cpp is build 12119, `-ncmoe 29 -ub 2048 -b 2048 -fa 1` (28 runs out of VRAM).

Benchmark tools (RustingStrata bench vs llama-bench):

* Decode, short prompt: 70 tok/s vs 43
* Decode after a 4K prompt: 65 vs 43
* Prefill at 4K: \~1870 vs 1210
* 32K context: 1470 prefill / 49 decode vs 1030 / 29
* With --mtp 1: up to 96 (86 after 4K), greedy

Real chat requests through both servers (same prompts, 256 tokens out, thinking off):

* No MTP: \~50 tok/s vs 37
* With MTP on both: \~62 vs 45-56
* 6K prompt prefill: 1320 vs 700

MTP works in current llama.cpp (`--spec-type draft-mtp`), \~63% of drafts accepted and +25% decode on this file. RustingStrata is still ahead with MTP on both sides.

Vulkan decodes at \~60 (85 with MTP), prefill is slower than CUDA on this file (\~1000-1260 at 4K).

KV cache is f16.

Code is on GitHub, free to use and I will be happy to get some feedback from AMD owners: [RustingStrata](https://github.com/RustingStudio/RustingStrata)

document fetched 2026-11-09 00:36 · canonical https://reddit.com/r/LocalLLaMA/comments/1xr2v59/rustingstrata_running_qwen3635ba3b_with_q5_and