strix-llama new branch: prompt processing +21–54% over Vulkan and up to 2.7× over stock ROCm kernel at 128k, same outputs quality
retained text
I've been tuning llama.cpp on my Strix Halo box (Radeon 8060S, 128 GB) and the numbers are good enough to share. It's a branch called `rdna35-perf` on top of [halo-box/strix-llama.cpp](https://github.com/halo-box/strix-llama.cpp), a llama.cpp fork aimed at Strix Halo: [**https://github.com/asf0/strix-llama.cpp/tree/rdna35-perf**](https://github.com/asf0/strix-llama.cpp/tree/rdna35-perf) (10 commits, ROCm/HIP only) **TL;DR** * **Prompt processing:** 21–54% faster than Vulkan, and 8–174% faster than stock ROCm. The lead over ROCm grows with context, because stock ROCm flash attention falls off hard at depth. * **Generation:** ties Vulkan on the dense 27B. On the MoE and the 2B it's 2–3% behind Vulkan at short context and within 1% from 64k up. 5–29% faster than stock ROCm. * **Quality:** outputs stay closer to stock ROCm than Vulkan's do, and perplexity is unchanged. |Model|PP vs Vulkan (0k → 128k)|PP vs stock ROCm (0k → 128k)|TG vs Vulkan (0k → 128k)| |:-|:-|:-|:-| |Qwen3.6 35B-A3B (MoE)|\+54% → +36%|\+38% → +88%|−3% → 0%| |Qwen3.8 27B (dense)|\+38% → +25%|\+8% → +174%|0% → 0%| |Qwen3.5 2B (dense)|\+21% → +34%|\+9% → +87%|−3% → 0%| # Results llama-bench, Q8\_0 weights, q8\_0 KV cache, flash attention on, tok/s. "Depth" is how many tokens are already in the KV cache when the test runs. Vulkan and ROCm are the same strix-llama.cpp master that rdna35-perf is built on. **Qwen3.6 35B-A3B** (MoE, ub 4096) |Test|Depth|Vulkan|ROCm|**rdna35-perf**|**vs Vulkan**|**vs ROCm**| |:-|:-|:-|:-|:-|:-|:-| |pp4096|0|2625|2935|**4045**|\+54%|\+38%| |pp4096|8k|2304|2430|**3452**|\+50%|\+42%| |pp4096|16k|2065|1966|**3004**|\+45%|\+53%| |pp4096|32k|1716|1460|**2388**|\+39%|\+64%| |pp4096|64k|1279|959|**1695**|\+32%|\+77%| |pp4096|128k|786|568|**1068**|\+36%|\+88%| |tg64|0|58.2|52.3|**56.3**|−3%|\+8%| |tg64|8k|56.1|50.4|**54.3**|−3%|\+8%| |tg64|16k|54.6|48.5|**53.2**|−3%|\+10%| |tg64|32k|51.8|45.3|**50.7**|−2%|\+12%| |tg64|64k|47.1|39.7|**46.4**|−1%|\+17%| |tg64|128k|39.9|32.2|**39.9**|0%|\+24%| **Qwen3.8 27B** (dense, ub 2048) |Test|Depth|Vulkan|ROCm|**rdna35-perf**|**vs Vulkan**|**vs ROCm**| |:-|:-|:-|:-|:-|:-|:-| |pp2048|0|484|614|**666**|\+38%|\+8%| |pp2048|8k|442|542|**611**|\+38%|\+13%| |pp2048|16k|421|476|**575**|\+37%|\+21%| |pp2048|32k|385|348|**513**|\+33%|\+47%| |pp2048|64k|326|204|**422**|\+29%|\+107%| |pp2048|128k|250|114|**312**|\+25%|\+174%| |tg64|0|8.0|7.6|**8.0**|0%|\+5%| |tg64|8k|7.9|7.4|**7.9**|0%|\+7%| |tg64|16k|7.8|7.2|**7.8**|0%|\+9%| |tg64|32k|7.6|6.8|**7.6**|0%|\+12%| |tg64|64k|7.3|6.2|**7.3**|0%|\+18%| |tg64|128k|6.8|5.2|**6.7**|0%|\+29%| **Qwen3.5 2B** (dense, ub 2048) |Test|Depth|Vulkan|ROCm|**rdna35-perf**|**vs Vulkan**|**vs ROCm**| |:-|:-|:-|:-|:-|:-|:-| |pp2048|0|6481|7179|**7852**|\+21%|\+9%| |pp2048|8k|5618|6253|**7158**|\+27%|\+14%| |pp2048|16k|5000|5172|**6611**|\+32%|\+28%| |pp2048|32k|4155|3888|**5580**|\+34%|\+44%| |pp2048|64k|3220|2597|**4277**|\+33%|\+65%| |pp2048|128k|2160|1547|**2886**|\+34%|\+87%| |tg64|0|99.3|91.5|**96.8**|−3%|\+6%| |tg64|8k|95.9|88.2|**93.2**|−3%|\+6%| |tg64|16k|92.9|84.5|**91.4**|−2%|\+8%| |tg64|32k|88.3|79.1|**87.1**|−1%|\+10%| |tg64|64k|79.5|69.2|**79.2**|0%|\+14%| |tg64|128k|67.6|56.2|**67.7**|0%|\+21%| f16 KV gives the same picture: PP 20–56% faster than Vulkan and up to +171% over stock ROCm at 128k, TG within −3% / +2% of Vulkan. # Quality rdna35-perf changes kernels, so I checked it doesn't change outputs: KL divergence against stock ROCm on wikitext-2 (6 × 4096-token chunks), plus perplexity at 32k context. |Model|KLD vs stock ROCm (Vulkan / rdna35-perf)|Same top token (Vulkan / rdna35-perf)|PPL @ 32k (Vulkan / ROCm / rdna35-perf)| |:-|:-|:-|:-| |Qwen3.6 35B-A3B|0.0038 / **0.0018**|97.8% / **98.5%**|6.028 / 6.023 / 6.024| |Qwen3.8 27B|0.0066 / **0.0013**|98.9% / **99.7%**|6.003 / 6.004 / 6.007| |Qwen3.5 2B|0.0008 / **0.0001**|98.6% / **99.5%**|10.046 / 10.049 / 10.049| rdna35-perf is closer to stock ROCm than Vulkan is, and perplexity is the same across all three builds. # What changed * **Flash attention kernel for RDNA3.5** (`fattn-rdna35`). This is the biggest win and the reason long-context PP holds up. `GGML_FA_RDNA35=0` turns it off if you want to compare. * **Prefill GEMM tuning** for the quantized matmuls. * **MoE:** routing helper, fused reduction + residual, grouped mat-vec for speculative-decoding verify batches. * **Gated DeltaNet** (the linear-attention layers in Qwen3.5/3.6/3.8): fused SiLU-gated output norm and fused decode. * **Decode:** plain mat-vec for rows longer than 4096 (this closed the 27B decode gap to Vulkan), plus decode fusions for the Qwen3.6 MoE. * **Misc:** HIP graphs off by default on RDNA3.5 (slower here), a faster penalties sampler, host-side prefill for Qwen3.5/3.6 with a draft model, and DFlash drafts read target features in place on the GPU. I have only tried those 3 models so far.
document fetched 2026-10-13 00:43 · canonical https://reddit.com/r/LocalLLaMA/comments/1x3s4qu/strixllama_new_branch_prompt_processing_2154_over