← sources

llama : add a GPU cache for MoE experts kept in host memory by am17an · Pull Request #29887 · ggml-org/llama.cpp

source: reddit (reddit:LocalLLaMA)published 2026-10-07 18:15ingested 2026-10-08 02:41

retained text

Potentially big speedup for MoE models that don’t fully fit in VRAM.

Are you GPU Poor? Show your speedups ;)

document fetched 2026-10-08 00:43 · canonical https://reddit.com/r/LocalLLaMA/comments/1x03xkc/llama_add_a_gpu_cache_for_moe_experts_kept_in