← sources

Pre-caching context overhead (system prompt, tools, skills, ...) ?

source: reddit (reddit:LocalLLaMA)published 2026-09-20 08:02ingested 2026-09-21 02:38

retained text

I run pretty large models (DS4 Flash, Qwen3.8 Flash, ...) on a machine with pretty slow prompt processing (M2 Ultra).

The first prompt takes a good 2-3 minutes to process, because of all the overhead the harness passes. Once that is done, context is cached and the following turns are much faster.

  
Since the overhead is always the same for a given project, can't I just program the harness to submit the overhead as soon as I open the project? This gives the machine time to process it while I type my first prompt.

I use Opencode FWIW.

document fetched 2026-09-21 00:37 · canonical https://reddit.com/r/LocalLLaMA/comments/1wlazxc/precaching_context_overhead_system_prompt_tools