← sources

Qwen Flash next on 64GB RAM unified iGPU anyone ? (non-mac)

source: reddit (reddit:LocalLLaMA)published 2026-10-03 13:43ingested 2026-10-04 02:38

retained text

I've seen people reporting running it with 12GB VRAM + 64 GB RAM. Also with 64 GB RAM unified in Macs, but I was wondering if it's possible with any inference backend to run it for example on a 64 GB RAM minipc+ iGPU (780m in my case).  
I'm currently running 125B Ling 3.0 flash at Q2 quants with llama.cpp (vulkan), its relatively usable, so I was wondering if a similar quant of Qwen Flash Next with the ngrams offloaded to SSD could work (even at low token/s). As far as I know this can't be done with llama.cpp now. Other inference engines does not seem to work with vulkan.

EDIT: Thanks everyone! It's working with the Q2 quant Qwen3.8-Flash-Next-GSQ-RCO-GGUF using llama.cpp with `-lm mmap --lazy-mode on`

document fetched 2026-10-04 00:36 · canonical https://reddit.com/r/LocalLLaMA/comments/1wwnm5x/qwen_flash_next_on_64gb_ram_unified_igpu_anyone