← sources

vulkan: int8 coopmat1 matmul implementation for AMD RDNA3 and RDNA4 (… · ggml-org/llama.cpp@70c4e15

source: reddit (reddit:LocalLLaMA)published 2026-09-24 13:37ingested 2026-09-25 02:42

retained text

This has made a massive improvement in performance on my 7900XTX

before:

```
| model                          |       size |     params | backend    | ngl | n_ubatch |  fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | --: | --------------: | -------------------: |
| gemma4 26B.A4B Q4_0            |  13.26 GiB |    25.23 B | Vulkan     |  -1 |     1024 |   1 |           pp512 |      3410.53 ± 22.72 |
| gemma4 26B.A4B Q4_0            |  13.26 GiB |    25.23 B | Vulkan     |  -1 |     1024 |   1 |           tg128 |        135.64 ± 0.80 |
| gemma4 26B.A4B Q4_0            |  13.26 GiB |    25.23 B | Vulkan     |  -1 |     1024 |   1 |   pp512 @ d8192 |      2477.27 ± 76.78 |
| gemma4 26B.A4B Q4_0            |  13.26 GiB |    25.23 B | Vulkan     |  -1 |     1024 |   1 |   tg128 @ d8192 |        120.42 ± 0.46 |
| gemma4 26B.A4B Q4_0            |  13.26 GiB |    25.23 B | Vulkan     |  -1 |     1024 |   1 |  pp512 @ d16384 |      2130.36 ± 35.16 |
| gemma4 26B.A4B Q4_0            |  13.26 GiB |    25.23 B | Vulkan     |  -1 |     1024 |   1 |  tg128 @ d16384 |        118.42 ± 0.14 |
```

after:

```
| model                          |       size |     params | backend    | ngl | n_ubatch |  fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | --: | --------------: | -------------------: |
| gemma4 26B.A4B Q4_0            |  13.26 GiB |    25.23 B | Vulkan     |  -1 |     1024 |   1 |           pp512 |     4331.21 ± 132.67 |
| gemma4 26B.A4B Q4_0            |  13.26 GiB |    25.23 B | Vulkan     |  -1 |     1024 |   1 |           tg128 |        142.76 ± 0.92 |
| gemma4 26B.A4B Q4_0            |  13.26 GiB |    25.23 B | Vulkan     |  -1 |     1024 |   1 |   pp512 @ d8192 |      2885.67 ± 80.24 |
| gemma4 26B.A4B Q4_0            |  13.26 GiB |    25.23 B | Vulkan     |  -1 |     1024 |   1 |   tg128 @ d8192 |        125.29 ± 0.14 |
| gemma4 26B.A4B Q4_0            |  13.26 GiB |    25.23 B | Vulkan     |  -1 |     1024 |   1 |  pp512 @ d16384 |      2402.13 ± 44.16 |
| gemma4 26B.A4B Q4_0            |  13.26 GiB |    25.23 B | Vulkan     |  -1 |     1024 |   1 |  tg128 @ d16384 |        120.82 ± 0.18 |
```

document fetched 2026-09-25 00:45 · canonical https://reddit.com/r/LocalLLaMA/comments/1wp1vex/vulkan_int8_coopmat1_matmul_implementation_for