# Two ~300B MoE models, each on ONE 128 GB mini PC (AMD Strix Halo): GLM-5.3-Flash at ~580 tok/s prefill, MiMo-V2.6-Flash up to 44 tok/s decode. EXL3 weights + open ROCm engine — retained record

> Retained reddit record from reddit:LocalLLaMA.

Canonical: https://highsignal.app/data/records/54ebce8a676daf3e

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:LocalLLaMA)  published 2026-10-03 14:16 ingested 2026-10-04 02:38

[original source ↗](https://www.reddit.com/r/LocalLLaMA/comments/1wwocik/two_300b_moe_models_each_on_one_128_gb_mini_pc/)

## retained text



```
We built an engine, Kyojin, on top of ExLlamaV3 for Strix Halo (gfx1151, ROCm), and packed two 300B-class MoE models so each fits one 128 GB machine. First release, all measured on Ryzen AI Max+ 395.

|Model|GLM-5.3-Flash|MiMo-V2.6-Flash-MOPD|
|:-|:-|:-|
|Size|99.7 GB|105 GB|
|Prefill|580 tok/s at 3.5K, 546 at 64K|about 650 tok/s at 4K|
|Decode|26 to 30 tok/s (MTP)|32 prose / 35 chat / 44 code (speculative), 29 plain|
|KLD vs official FP8|0.151|0.0713|
|Top-1 agreement with FP8|89.3 %|92.0 %|

**Where the weights come from.** MiMo is our own quantisation. The GLM pack mixes turboderp's public 2.05 and 3.05 bpw EXL3 tensors, with our layer mix and a small tuning stage. On the same 129 rows, his 2.05 bpw pack (85 GB) gets KLD 0.275; our mix (100 GB) gets 0.190. His is smaller and decodes about 10 % faster.

**Uncensored variants.** Separate `-Uncensored` repos: same weights plus one small file the engine applies at load, one switch turns it off.

**Not measured yet.** Task-suite scores for MiMo, GLM at 128K context, any GPU other than gfx1151. The conversion pipeline stays private.

**Quickstart.** Clone, `./build.sh`, `hf download yamz-labs/GLM-5.3-Flash-EXL3-Yamz`, `python tools/glm/serve.py --model ./glm-pack -c 131072 --num-draft 2`. You get an OpenAI-style API.

Models: [https://huggingface.co/yamz-labs](https://huggingface.co/yamz-labs)

Engine: [https://github.com/Yamz-Labs/kyojin](https://github.com/Yamz-Labs/kyojin)

Built on turboderp's ExLlamaV3, with ROCm work from sdougbrown and vcruz305.

If you own a Strix Halo machine, we'd love to see your tok/s. Issues, benchmarks and PRs are all welcome. Which model should we do next?

  

```



document fetched 2026-10-04 00:36 · canonical https://reddit.com/r/LocalLLaMA/comments/1wwocik/two_300b_moe_models_each_on_one_128_gb_mini_pc
