← sources

If your local LLM setup puts a timestamp at the top of the system prompt, your GPU re-reads the whole prompt every message. Two curls show it.

source: reddit (reddit:selfhosted)published 2026-10-12 08:02ingested 2026-10-13 02:39

retained text

This is about Ollama, but the mechanism is the same for anything built on llama.cpp (LM Studio, Jan, Open WebUI in front of Ollama, your own scripts).

A local model keeps a KV cache of the prompt it has already processed, and reuses it on the next request. That reuse only survives on a **shared prefix**: it holds for exactly as long as the new prompt matches the old one from the very first token. The first token that differs throws away everything after it, however identical the rest is.

Which means one small thing that changes per message, sitting near the top of your system prompt, costs you the entire prompt on every single turn. Timestamps are the usual culprit. "Current date and time: 2026-10-12 09:41" is a very common first line.

**Measured.** Same model (qwen2.5:7b), same 324-token system prompt, same words. The only difference is where one 21-token timestamp sits:

| Timestamp position | Prompt tokens | Reused from cache | Prefill time |
|---|---|---|---|
| At the top | 324 | 4 | 64.6 ms |
| At the bottom | 324 | 290 | 18.6 ms |

3.5x the prefill, same prompt. On a 2,000-token system prompt with RAG chunks the multiplier is worse, and prefill is the part of each turn where the GPU does real work.

**Check your own setup in two calls.** Since Ollama 0.33.3 the native API reports what the cache gave back. Send the same chat twice and look at the second answer:

```sh
for i in 1 2; do
  curl -s localhost:11434/api/chat -d '{
    "model": "qwen2.5:7b", "stream": false,
    "messages": [{"role":"system","content":"<paste your real system prompt here>"},
                 {"role":"user","content":"hi"}],
    "options": {"num_predict": 1}
  }' | jq '{prompt_eval_count, prompt_eval_cached_count, prompt_eval_duration}'
done
```

On the second call `prompt_eval_cached_count` is how much the cache kept. If it is near the total, your prompt is stable. If it is a handful of tokens, find the first thing in your prompt that changed between calls; everything after it was recomputed.

**The fix costs nothing.** Order, not content. Stable material first: role, rules, style, examples. Anything that changes per turn goes last: time, user name, session id, retrieved chunks. And keep RAG chunks in a stable order when the set has not changed; re-ranking them between turns breaks the cache at the first chunk that moved. The model is told the same things. It just does not re-read them.

**Two traps while you measure:**

- `prompt_eval_duration` only times the *uncached* tokens since 0.33.3, while `prompt_eval_count` is still the total. Dividing one by the other, the way every dashboard does, inflates prefill speed by the cache hit ratio. Divide by `prompt_eval_count - prompt_eval_cached_count`.
- Ollama 0.35+ ships a ready-made `timings.prompt_per_second` on the OpenAI-compatible endpoint with the same mistake baked in. I measured 3875 tok/s stated against 125 real on a warm cache (filed as ollama/ollama#18948). Don't trust that field on a warm cache.

---

Disclosure: I maintain [LLMxRay](https://github.com/LogneBudo/llmxray), a local observability UI for Ollama (Apache-2.0, nothing leaves localhost, `npx llmxray`). It has a page that finds the values in your prompt that change between turns, shows where the shared prefix breaks, and measures what moving them saves against your own daemon. Everything above works without it; the two curls are the whole method.

Curious what prompt sizes people here run with Open WebUI and friends, and whether anyone has looked at what their front end puts at the top of the prompt. Several of them inject the date first.

document fetched 2026-10-13 00:31 · canonical https://reddit.com/r/selfhosted/comments/1x3w0gd/if_your_local_llm_setup_puts_a_timestamp_at_the