cuda, hip: GDN chunked kernel by am17an · Pull Request #29353 · ggml-org/llama.cpp
retained text
|Performance on|Model|Prompt tokens|master tok/s|pr tok/s|Change| |:-|:-|:-|:-|:-|:-| |RTX PRO 6000 Blackwell|Qwen3.5-35B-A3B-Q8\_0|2048|13,653.58|16,459.56|\+20.6%| |RTX PRO 6000 Blackwell|Qwen3.5-35B-A3B-Q8\_0|4096|14,090.26|16,595.47|\+17.8%| |RTX PRO 6000 Blackwell|Qwen3.8-27B-Q8\_0|2048|4,470.39|4,757.20|\+6.4%| |RTX PRO 6000 Blackwell|Qwen3.8-27B-Q8\_0|4096|4,442.45|4,699.08|\+5.8%| |Spark GB10|Qwen3.6-35B-A3B-UD-Q4\_K\_M|2048|3,271.18|3,406.68|\+4.1%| |Spark GB10|Qwen3.6-35B-A3B-UD-Q4\_K\_M|4096|3,254.79|3,366.88|\+3.4%| |Spark GB10|Qwen3.8-27B-UD-Q8\_K\_L|2048|942.84|984.86|\+4.5%| |Spark GB10|Qwen3.8-27B-UD-Q8\_K\_L|4096|928.16|973.34|\+4.9%| |Spark GB10|Qwen3.8-Flash-Next-UD-IQ4\_XS (`-lzm on`)|2048|1,044.71|1,104.31|\+5.7%| |Spark GB10|Qwen3.8-Flash-Next-UD-IQ4\_XS (`-lzm on`)|4096|1,068.70|1,124.01|\+5.2%| |RTX 5090|Qwen3.5-35B-A3B-Q4\_K\_S|2048|12,477.32|14,524.81|\+16.4%| |RTX 5090|Qwen3.5-35B-A3B-Q4\_K\_S|4096|12,978.57|15,298.00|\+17.9%| |RTX 5090|Qwen3.8-27B-Q4\_K\_M|2048|4,095.61|4,320.75|\+5.5%| |RTX 5090|Qwen3.8-27B-Q4\_K\_M|4096|4,114.11|4,353.09|\+5.8%| |RTX 4090|Qwen3.5-35B-A3B-Q4\_K\_S|2048|10,872.34|12,044.25|\+10.8%| |RTX 4090|Qwen3.5-35B-A3B-Q4\_K\_S|4096|11,187.34|12,428.05|\+11.1%| |RTX 4090|Qwen3.8-27B-Q4\_K\_M|2048|3,336.69|3,438.65|\+3.1%| |RTX 4090|Qwen3.8-27B-Q4\_K\_M|4096|3,340.90|3,446.88|\+3.2%| |RTX 3090|Qwen3.5-35B-A3B-Q4\_K\_S|2048|5,672.07|6,061.59|\+6.9%| |RTX 3090|Qwen3.5-35B-A3B-Q4\_K\_S|4096|5,505.49|5,683.79|\+3.2%| |RTX 3090|Qwen3.8-27B-Q4\_K\_M|2048|1,400.46|1,412.40|\+0.9%| |RTX 3090|Qwen3.8-27B-Q4\_K\_M|4096|1,337.41|1,367.46|\+2.2%|
document fetched 2026-10-15 00:46 · canonical https://reddit.com/r/LocalLLaMA/comments/1x5skta/cuda_hip_gdn_chunked_kernel_by_am17an_pull