← sources

AMD Loom Fun: Optimizing the humble Qwen3.5-4B Q4_K_M on Ryzen AI 375 (890M / gfx1150)

source: reddit (reddit:LocalLLaMA)published 2026-11-18 20:56ingested 2026-11-19 02:39

retained text

Qwen3.5-4B is a popular choice for local AI apps that run on laptops with 32 GB or less RAM. I've been playing with AMD's new Loom inference stack lately and wanted to see what it could do for this model, since speedup would make a big difference in responsiveness. 

I ran the `/loom-optimize` skill on a gfx1150 32 GB RAM laptop and the results are pretty good! Up to 2.6x improvement on prefill and up to 1.3x on decode.

**Qwen3.5-4B (Q4_K_M) on a Radeon 890M (Ubuntu 26.04, gfx1150, Strix Point laptop, 32 GB LPDDR5X-7500)**

| Metric (tok/s) | Loom | llama.cpp ROCm | llama.cpp Vulkan | Loom vs ROCm | Loom vs Vulkan |
|---|---:|---:|---:|---:|---:|
| Decode (short prompt) | **31.1** | 21.7 | 21.1 | 1.43× | 1.47× |
| Decode @ 2k context | **28.2** | 24.0 | 21.2 | 1.18× | 1.33× |
| Decode @ 32k context | **22.4** | 17.2 | 14.5 | 1.30× | 1.54× |
| Prefill 1k | **996** | 703 | 608 | 1.42× | 1.64× |
| Prefill 2k | **1016** | 703 | 641 | 1.44× | 1.59× |
| Prefill 8k | **1021** | 625 | 632 | 1.63× | 1.62× |
| Prefill 16k | **1041** | 497 | 583 | 2.09× | 1.78× |
| Prefill 32k | **929** | 357 | 495 | 2.60× | 1.88× |
| 4 concurrent streams (total) | **82.3** | 74.4 | 69.4 | 1.11× | 1.19× |

Model card with quick start and perf results for gfx1150, gfx1151 (strix halo), and gfx1201 (Radoen 9700/9070): https://huggingface.co/loom-fun/unsloth-Qwen3.5-4B-loom

GitHub: https://github.com/loom-fun/loom-skills

What do you all think, does faster inference on small models unlock something interesting for building apps?

document fetched 2026-11-19 00:42 · canonical https://reddit.com/r/LocalLLaMA/comments/1xzqll4/amd_loom_fun_optimizing_the_humble_qwen354b_q4_k