← sources

preserve_thinking off is FINE

source: reddit (reddit:LocalLLaMA)published 2026-11-03 13:07ingested 2026-11-04 02:41

retained text

Let's take a break from all the engine, model and speed posts and discuss something else:

Qwen 3.8 likes to think. A lot. We all know that, and the context cost you pay for that is MASSIVE when you enable preserve thinking. Even with a Swift finetune, there are so many tokens you keep in context over a longer session. Often, easily over 50% of the KV cache is just thinking traces.

And for what use? I found: Next to nothing! The model can handle itself just fine without knowing HOW it reached a decision, and instead only looking at what it did in the end. Replies and toolcalls: it's all there.

I added an AGENTS.md instruction that when it finds a gotcha, something unintuitive, something it only reached after lots of trial and error, it should document that in a memory file to mitigate the missing past-thinking and it works beautifully.

Results (my subjective impressions):

* Problem solving capabilities haven't suffered much if at all
* I get neat, permanent notes on issues with gotchas
* The dreaded, context-destroying compactions have reduced by at least 2/3rds in your average sessions, the cache holds for so much longer


So let's discuss: preserve_thinking on or off. Your experiences?

document fetched 2026-11-04 00:41 · canonical https://reddit.com/r/LocalLLaMA/comments/1xmsgz5/preserve_thinking_off_is_fine