# CUDA: fuse MoE gate/up matmuls and GLU in MMQ by anujj · Pull Request #31188 · ggml-org/llama.cpp — retained record

> Retained reddit record from reddit:LocalLLaMA.

Canonical: https://highsignal.app/data/records/0adec899a3fe8434

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:LocalLLaMA)  published 2026-10-28 13:40 ingested 2026-10-29 02:42

[original source ↗](https://www.reddit.com/r/LocalLLaMA/comments/1xhodj3/cuda_fuse_moe_gateup_matmuls_and_glu_in_mmq_by/)

## retained text



```
download your CUDA/MoE prompt-processing speedup today:

|GPU|model|master|PR||
|:-|:-|:-|:-|:-|
|DGX Spark (GB10)|Qwen3-30B-A3B Q4\_K\_M|2911.4|3410.8|\+17.2%|
||Qwen3.6-35B-A3B Q4\_K\_M|3235.8|3556.7|\+9.9%|
||Qwen3.6-35B-A3B UD-Q4\_K\_M|3264.7|3561.8|\+9.1%|
|RTX 5090|Qwen3-30B-A3B Q4\_K\_M|12611.7|13330.3|\+5.7%|
||Qwen3.6-35B-A3B Q4\_K\_M|15000.3|15852.0|\+5.7%|
||Qwen3.6-35B-A3B UD-Q4\_K\_M|15030.2|15947.8|\+6.1%|
```



document fetched 2026-10-29 00:44 · canonical https://reddit.com/r/LocalLLaMA/comments/1xhodj3/cuda_fuse_moe_gateup_matmuls_and_glu_in_mmq_by
