CUDA: fuse MoE gate/up matmuls and GLU in MMQ by anujj · Pull Request #31188 · ggml-org/llama.cpp
retained text
download your CUDA/MoE prompt-processing speedup today: |GPU|model|master|PR|| |:-|:-|:-|:-|:-| |DGX Spark (GB10)|Qwen3-30B-A3B Q4\_K\_M|2911.4|3410.8|\+17.2%| ||Qwen3.6-35B-A3B Q4\_K\_M|3235.8|3556.7|\+9.9%| ||Qwen3.6-35B-A3B UD-Q4\_K\_M|3264.7|3561.8|\+9.1%| |RTX 5090|Qwen3-30B-A3B Q4\_K\_M|12611.7|13330.3|\+5.7%| ||Qwen3.6-35B-A3B Q4\_K\_M|15000.3|15852.0|\+5.7%| ||Qwen3.6-35B-A3B UD-Q4\_K\_M|15030.2|15947.8|\+6.1%|
document fetched 2026-10-29 00:44 · canonical https://reddit.com/r/LocalLLaMA/comments/1xhodj3/cuda_fuse_moe_gateup_matmuls_and_glu_in_mmq_by