# MoE offload on a 3090: prefill 345 -> 466 t/s by tuning cudaMemcpy (Qwen3.6 35B-A3B) — retained record

> Retained reddit record from reddit:LocalLLaMA.

Canonical: https://highsignal.app/data/records/f12fbe24ebe74c64

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:LocalLLaMA)  published 2026-11-15 17:43 ingested 2026-11-16 02:38

[original source ↗](https://www.reddit.com/r/LocalLLaMA/comments/1xx1owx/moe_offload_on_a_3090_prefill_345_466_ts_by/)

## retained text



```
https://i.redd.it/5dlnj6wict1i1.gif

    I made a tool that tunes copies that speeds up CPU offloading.
    
    1 3090 + 7800X3D
    
    llama-bench -m Qwen3.6-35B-A3B-Q8_0.gguf -ngl 99 -fa 1 -ot exps=CPU -p 2048 -n 0 -r 3
    
    cmt tune -- llama-bench -m Qwen3.6-35B-A3B-Q8_0.gguf -ngl 99 -fa 1 -ot exps=CPU -p 2048 -n 0 -r 3
    
    before: prefill 345 t/s
    after: prefill 466 t/s
    
    
    blog: https://maknee.github.io/blog/2026/CudaMemcpy/
    code: https://github.com/Maknee/cudamemcpy-tuner
```



document fetched 2026-11-16 00:37 · canonical https://reddit.com/r/LocalLLaMA/comments/1xx1owx/moe_offload_on_a_3090_prefill_345_466_ts_by
