← sources

PSA: llama.cpp -cram should be increased for agentic workflows (default is 8192)

source: reddit (reddit:LocalLLaMA)published 2026-09-24 15:44ingested 2026-09-25 02:42

retained text

Just a quick PSA. llama.cpp does have prompt caching. if you are running large context lengths and have long multiturn projects, increasing -cram can provide you with massive speedups. There is a point where context lengths can get so large that 8192mb is not enough and the whole context needs to be re processed again on every turn. personally, I have found 20480 to work well with Qwen 27B 3.8 at 262K context.

the main downside is this uses more ram. vram usage doesnt increase.

document fetched 2026-09-25 00:45 · canonical https://reddit.com/r/LocalLLaMA/comments/1wp5668/psa_llamacpp_cram_should_be_increased_for_agentic