← sources

llama.cpp: Possible to use Vulkan/iGPU and lazy-mode?

source: reddit (reddit:LocalLLaMA)published 2026-11-14 08:27ingested 2026-11-15 02:36

retained text

With llama.cpp pure CPU mode and "--lazy-mode on" one can use larger than memory models, like Qwen3.8-Flash-Next and on-demand SSD streaming - slow but working.

There is a nice iGPU with unified memory that can be used easily via Vulkan. However, it seems to be impossible to split a large model - use as much as fits into memory with Vulkan and then stream the rest in via SSD and use CPU. --list-devices shows only one Vulkan device and --ngl is of no use -> Vulkan always runs out of memory with lazy-mode on - all of the model is tried to allocated via Vulkan?

Is this possible at all?

document fetched 2026-11-15 00:36 · canonical https://reddit.com/r/LocalLLaMA/comments/1xvxbny/llamacpp_possible_to_use_vulkanigpu_and_lazymode