# MoE Offloading update Exllamav3 on 9070XT — retained record

> Retained reddit record from reddit:LocalLLaMA.

Canonical: https://highsignal.app/data/records/435b658d4f03e49a

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:LocalLLaMA)  published 2026-11-17 11:10 ingested 2026-11-18 02:42

[original source ↗](https://www.reddit.com/r/LocalLLaMA/comments/1xyhrnn/moe_offloading_update_exllamav3_on_9070xt/)

## retained text



```
This is a follow up on [https://www.reddit.com/r/LocalLLaMA/comments/1xun15v/moe\_and\_expert\_offloading\_on\_llama\_exllamav3/](https://www.reddit.com/r/LocalLLaMA/comments/1xun15v/moe_and_expert_offloading_on_llama_exllamav3/)

While we wait for Qwen 4, here is Qwen 3.6 35B A3B at Q4 on 9070XT 16GB with offloaded experts on DDR4 3200Mhz ram.

I've been tinkering still with the config for experts offloading and found substantial gains from my previous post.

While decode stayed roughly in line fro last time, increasing chunk size to 5120 saw prefill over 2000 t/s at all context lengths, topping at 2606 t/s @ 32K. *From \~1300t/s*  
GPU utilization increases with batch size as it allows for more parallel processing. I saw no gains over 5120. TabbyAPI's documented ideal range is 512–4096, so your miles might vary.

Exllama flags like cpu\_threads 6 also made a small improvement in decode.  
Swizzle, gdn\_sub, bg\_gdn and wide flags were all regressions. (only tested isolated 1by1).

cpu\_moe\_pinned\_arena caused some core dumps, and the one run I managed to complete had a small gain. 

Vision have been turned off and offloaded.

*I still have not found an .exl3 quant for the Coder version of Qwen 3.8 Flash Next, and the qfn version I can run leaves me with 8K context with almost no vram or system ram left to spare.*

Results and more graphs and tables are at [https://github.com/willeyh-git/moe-offload-bench](https://github.com/willeyh-git/moe-offload-bench)  
All engines were compiled for ROCm on my system.


```



document fetched 2026-11-18 00:46 · canonical https://reddit.com/r/LocalLLaMA/comments/1xyhrnn/moe_offloading_update_exllamav3_on_9070xt
