AMD Loom Fun: Added Radeon 9700 support over the weekend. Benchmarked 48 models with an average 42% speedup over llama.cpp.
retained text
On Friday I [posted about AMD Loom](https://www.reddit.com/r/LocalLLaMA/s/4RHJh9XrzK), a new experimental inference stack that includes a runtime, kernel compiler, and inference server. People can use the entire thing as a vertical stack, or the compiler/runtime can be integrated into any other inference project (I'm working on Gufo right now, the team is also working on a llamacpp backend). I started with Strix Halo, where I ported the top-51 trending Q4\_K\_M LLMs from Hugging Face and got an average of 40% speedup over llama.cpp across decode and prefill metrics (picking the best results from ROCm or Vulkan backends). I'm back today with Radeon 9700 results. I took the same 51 LLMs, cut it down to the 48 that fit in 32 GB of VRAM, and got an average of 42% speedup. Some of the data is posted to the table below, and all data is available to browse on [https://loom-fun.github.io/loom-dev/index.html?gpu=radeon-r9700](https://loom-fun.github.io/loom-dev/index.html?gpu=radeon-r9700) All kernels were generated via agent skills: * loom-skills repo: [https://github.com/loom-fun/loom-skills](https://github.com/loom-fun/loom-skills) * deterministic SDK (runtime, compiler, server): [https://github.com/ROCm/hrx-system](https://github.com/ROCm/hrx-system) |Model|Loom pp32k (tok/s)|llama.cpp pp32k (tok/s)|Loom short decode (tok/s)|llama.cpp short decode (tok/s)| |:-|:-|:-|:-|:-| |[Flint](https://huggingface.co/loom-fun/ggml-org-Flint-loom)|6,476|3,190|255|152| |[gemma-3-1b-it](https://huggingface.co/loom-fun/ggml-org-gemma-3-1b-it-loom)|n/a|n/a|400|322| |[gemma-4-12B-it](https://huggingface.co/loom-fun/unsloth-gemma-4-12B-it-loom)|2,811|1,537|127|67.3| |[gemma-4-26B-A4B-it](https://huggingface.co/loom-fun/lmstudio-community-gemma-4-26B-A4B-it-loom)|doesn't fit (32 GB)|doesn't fit (32 GB)|151|72.2| |[gemma-4-31B-it](https://huggingface.co/loom-fun/unsloth-gemma-4-31B-it-loom)|doesn't fit (32 GB)|doesn't fit (32 GB)|62.9|43.2| |[gemma-4-E2B-it](https://huggingface.co/loom-fun/unsloth-gemma-4-E2B-it-loom)|14,289|14,519|272|234| |[gemma-4-E4B-it](https://huggingface.co/loom-fun/unsloth-gemma-4-E4B-it-loom)|7,021|6,324|192|173| |[gpt-oss-20b](https://huggingface.co/loom-fun/unsloth-gpt-oss-20b-loom)|5,319|4,368|195|182| |[HyperCLOVAX-SEED-Text-Instruct-1.5B](https://huggingface.co/loom-fun/rippertnt-HyperCLOVAX-SEED-Text-Instruct-1.5B-loom)|11,653|8,078|362|344| |[Laguna-XS-2.1](https://huggingface.co/loom-fun/ggml-org-Laguna-XS-2.1-loom)|doesn't fit (32 GB)|doesn't fit (32 GB)|102|—| |[LFM2.5-24B-A2B](https://huggingface.co/loom-fun/ggml-org-LFM2.5-24B-A2B-loom)|10,099|5,479|244|197| |[LFM3-1.7B](https://huggingface.co/loom-fun/bartowski-LFM3-1.7B-loom)|20,062|13,614|372|261| |[LFM3-Omni-2B](https://huggingface.co/loom-fun/ggml-org-LFM3-Omni-2B-loom)|20,220|13,435|368|299| |[Llama-3.2-3B-Instruct](https://huggingface.co/loom-fun/unsloth-Llama-3.2-3B-Instruct-loom)|5,687|3,908|228|211| |[Mellum2.1-12B-A2.5B-Thinking](https://huggingface.co/loom-fun/bartowski-Mellum2.1-12B-A2.5B-Thinking-loom)|8,822|5,486|224|149| |[MiniCPM-V-4.7-35B-A3B](https://huggingface.co/loom-fun/bartowski-MiniCPM-V-4.7-35B-A3B-loom)|5,717|3,540|135|113| |[Muse-Glimmer-30B](https://huggingface.co/loom-fun/ggml-org-Muse-Glimmer-30B-loom)|1,658|1,287|72.6|54.8| |[Nanbeige4.3-33B](https://huggingface.co/loom-fun/bartowski-Nanbeige4.3-33B-loom)|946|666|29.4|28.7| |[NVIDIA-Nemotron-3.5-Bolt-E4B-4GB](https://huggingface.co/loom-fun/nvidia-NVIDIA-Nemotron-3.5-Bolt-E4B-4GB-loom)|6,932|3,656|225|143| |[Ornith-1.0-35B](https://huggingface.co/loom-fun/lmstudio-community-Ornith-1.0-35B-loom)|5,628|3,320|152|127| |[Ornith-1.0-9B](https://huggingface.co/loom-fun/unsloth-Ornith-1.0-9B-loom)|4,886|3,391|100|88.0| |[Ornith-1.5-35B-A3B](https://huggingface.co/loom-fun/bartowski-Ornith-1.5-35B-A3B-loom)|5,586|3,136|145|78.8| |[Ornith-1.5-9B](https://huggingface.co/loom-fun/bartowski-Ornith-1.5-9B-loom)|4,767|3,156|151|90.5| |[Qwen2.5-0.5B-Instruct](https://huggingface.co/loom-fun/bartowski-Qwen2.5-0.5B-Instruct-loom)|26,022|17,812|646|529| |[Qwen2.5-1.5B-Instruct](https://huggingface.co/loom-fun/bartowski-Qwen2.5-1.5B-Instruct-loom)|12,546|8,908|360|331| |[Qwen2.5-32B-Instruct](https://huggingface.co/loom-fun/bartowski-Qwen2.5-32B-Instruct-loom)|doesn't fit (32 GB)|doesn't fit (32 GB)|30.0|29.2| |[Qwen2.5-7B-Instruct](https://huggingface.co/loom-fun/bartowski-Qwen2.5-7B-Instruct-loom)|3,810|2,679|118|116| |[Qwen2.5-VL-7B-Instruct](https://huggingface.co/loom-fun/ggml-org-Qwen2.5-VL-7B-Instruct-loom)|3,768|2,648|116|115| |[Qwen3-0.6B](https://huggingface.co/loom-fun/unsloth-Qwen3-0.6B-loom)|13,020|8,631|506|480| |[Qwen3-1.7B](https://huggingface.co/loom-fun/ggml-org-Qwen3-1.7B-loom)|10,313|7,014|316|297| |[Qwen3-14B](https://huggingface.co/loom-fun/ggml-org-Qwen3-14B-loom)|1,944|1,167|65.0|62.6| |[Qwen3-30B-A3B-Instruct-2507](https://huggingface.co/loom-fun/unsloth-Qwen3-30B-A3B-Instruct-2507-loom)|3,116|2,060|177|169| |[Qwen3-4B](https://huggingface.co/loom-fun/ggml-org-Qwen3-4B-loom)|3,770|2,826|231|130| |[Qwen3-4B-Instruct-2507](https://huggingface.co/loom-fun/unsloth-Qwen3-4B-Instruct-2507-loom)|3,795|2,856|180|166| |[Qwen3-8B](https://huggingface.co/loom-fun/unsloth-Qwen3-8B-loom)|3,076|2,287|163|94.7| |[Qwen3-Coder-30B-A3B-Instruct](https://huggingface.co/loom-fun/ggml-org-Qwen3-Coder-30B-A3B-Instruct-loom)|2,921|2,107|157|54.8| |[Qwen3-VL-4B-Instruct](https://huggingface.co/loom-fun/unsloth-Qwen3-VL-4B-Instruct-loom)|3,795|2,805|178|163| |[Qwen3-VL-8B-Instruct](https://huggingface.co/loom-fun/unsloth-Qwen3-VL-8B-Instruct-loom)|3,085|2,265|112|103| |[Qwen3.5-0.8B](https://huggingface.co/loom-fun/unsloth-Qwen3.5-0.8B-loom)|24,341|16,959|463|338| |[Qwen3.5-2B](https://huggingface.co/loom-fun/unsloth-Qwen3.5-2B-loom)|16,990|12,165|318|236| |[Qwen3.5-35B-A3B](https://huggingface.co/loom-fun/unsloth-Qwen3.5-35B-A3B-loom)|5,652|3,410|190|112| |[Qwen3.5-4B](https://huggingface.co/loom-fun/unsloth-Qwen3.5-4B-loom)|7,030|4,897|200|123| |[Qwen3.5-9B](https://huggingface.co/loom-fun/unsloth-Qwen3.5-9B-loom)|4,724|3,336|146|92.7| |[Qwen3.6-27B](https://huggingface.co/loom-fun/ggml-org-Qwen3.6-27B-loom)|1,503|1,164|60.1|38.9| |[Qwen3.6-35B-A3B](https://huggingface.co/loom-fun/ggml-org-Qwen3.6-35B-A3B-loom)|5,635|3,463|209|123| |[Qwen3.8-27B](https://huggingface.co/loom-fun/bartowski-Qwen3.8-27B-loom)|1,592|1,026|73.5|55.3| |[Sage-A1.5](https://huggingface.co/loom-fun/InternScience-Sage-A1.5-loom)|5,652|3,131|188|153| |[Spark-X2.5-4B](https://huggingface.co/loom-fun/XHToken-Spark-X2.5-4B-loom)|6,858|4,727|162|131| |[Spark-X2.6-4B](https://huggingface.co/loom-fun/bartowski-Spark-X2.6-4B-loom)|6,913|4,841|162|132|
document fetched 2026-11-17 00:42 · canonical https://reddit.com/r/LocalLLaMA/comments/1xxuzgg/amd_loom_fun_added_radeon_9700_support_over_the