# Strix Halo int4 cores: Gemma 4 31B at 2.43x llama.cpp, same quality — retained record

> Retained reddit record from reddit:LocalLLaMA.

Canonical: https://highsignal.app/data/records/3b537b9dc5b030fb

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:LocalLLaMA)  published 2026-11-17 10:08 ingested 2026-11-18 02:42

[original source ↗](https://www.reddit.com/r/LocalLLaMA/comments/1xygpmv/strix_halo_int4_cores_gemma_4_31b_at_243x/)

## retained text



```
The 8060S has an int4 x int4 WMMA instruction at 2x the FP16 rate (we measured 104 TOPS). Nothing in ROCm or llama.cpp uses it. We wrote a GEMM for it and run Gemma 4 with int4 weights and int4 activations. To maintain accuracy we use a combination of rotation and GPTQ, along with selective mixed precision.

llama-bench, Gemma 4 31B QAT Q4\_0 (llama.cpp b11056 Vulkan, -fa 1). Server benchmark, 2k-token prompts, greedy, single stream:

||pp512|pp2048|
|:-|:-|:-|
|llama.cpp|287|276|
|ours|679|724|

Full test sets, same settings, 31B:

||GSM8K|IFEval|MMLU-Pro|
|:-|:-|:-|:-|
|llama.cpp|96.2|92.2|85.1|
|ours|96.4|91.7|85.1|

Run it:

    curl -fsSL https://embedl.com/install.sh | sh && export PATH="$HOME/.local/bin:$PATH"
    embedl login
    embedl serve google/gemma-4-31B-it/ryzen-ai-max-395 -d
    embedl infer google/gemma-4-31B-it/ryzen-ai-max-395 --prompt "Explain mixture-of-experts in one sentence."
    embedl bench google/gemma-4-31B-it/ryzen-ai-max-395

12B, E4B and E2B are up too (2.80x / 1.58x / 1.39x end to end, in the chart). Swap the name, e.g. `google/gemma-4-12B-it/ryzen-ai-max-395`.

Model, methodology, and more: [https://www.embedl.com/google/gemma-4-31B-it/ryzen-ai-max-395](https://www.embedl.com/google/gemma-4-31B-it/ryzen-ai-max-395)
```



document fetched 2026-11-18 00:46 · canonical https://reddit.com/r/LocalLLaMA/comments/1xygpmv/strix_halo_int4_cores_gemma_4_31b_at_243x
