← sources

A failure mode I don't see discussed enough: AI quietly promotes its own inference into fact

source: reddit (reddit:PromptEngineering)published 2026-09-29 15:06ingested 2026-09-30 02:43

retained text

This isn't really about hallucinations.

I've noticed a different failure mode in long conversations:

You give the model incomplete information. It makes a reasonable assumption. Nothing wrong with that.

The problem is that 20 messages later, the assumption sometimes gets referred to as though it was something you originally told the model.

So I started treating **making an inference** and **promoting an inference into established context** as two separate things.

I suggest writing "remember this:" at the start..

    When something materially affects your answer, keep these separate:
    
    GIVEN — directly provided or verified.
    INFERRED — a reasonable conclusion, but not established.
    UNKNOWN — information you don't actually have.
    
    Do not silently turn INFERRED into GIVEN.
    
    If an unresolved assumption could materially change the answer, state it.
    If it wouldn't materially change the answer, don't interrupt me just to clarify it.
    
    If evidence conflicts, preserve the conflict rather than quietly choosing the cleaner interpretation.
    
    Otherwise answer normally. Don't turn this into a verbose checklist.

The last line matters. Without it, models sometimes respond by narrating every tiny uncertainty and become unusable.

The goal is:

**don't quietly turn “I think this is what you mean” into “this is what you told me”.**

I've found this especially useful for:

* long conversations
* planning
* debugging
* research
* business ideas
* reviewing documents
* advice where I've already formed an opinion

Interested to see your results. How did you improve it? I wanna see if you can break it, or, I should ask.. How did you break it?

**EDIT / ADDON — trying to recover an old chat from “context contamination”**

Something else that was brought up: once an incorrect assumption gets established in a long conversation, it can become strangely persistent.

You can correct it, the model acknowledges the correction, explains why it was wrong... and then a few messages later it starts reasoning from the same bad information again.

Sometimes starting a fresh chat immediately fixes the problem, which makes me suspect the issue isn’t always missing knowledge. The conversation itself has accumulated bad state — old assumptions, inferences treated as facts, incorrect attribution, superseded information, etc.

So I’ve been experimenting with a prompt specifically for rescuing an **existing conversation** rather than starting over:

# Context Contamination Recovery

"The current conversation may contain stale, incorrect, superseded, or contaminated context.

Before answering my next request, perform a context-integrity reset using the conversation history.

Classify any material information you intend to rely on as:

**GIVEN** — directly provided by me or explicitly present in a source.  
**VERIFIED** — independently checked against supporting evidence.  
**INFERRED** — reasonably concluded, but not directly established.  
**UNKNOWN** — unsupported, unresolved, or uncertain.  
**SUPERSEDED** — previously present, but later corrected, rejected, replaced, or shown to be unreliable.

Apply these rules:

* Never silently promote **INFERRED** or **UNKNOWN** information into **GIVEN** or **VERIFIED**.
* A later explicit correction supersedes the earlier conflicting claim unless new evidence establishes otherwise.
* Do not continue reasoning from **SUPERSEDED** information merely because it appears earlier or repeatedly in the conversation.
* Repetition does not increase evidential weight.
* Do not treat your own previous answers as independent evidence.
* Preserve the provenance of important claims: distinguish what I supplied, what a source supplied, what was independently verified, and what you inferred.
* Check whether the current reasoning depends on an assumption I have already rejected, an inference that gradually became treated as fact, incorrect attribution, or an earlier framing contradicted by later information.
* If it does, remove that dependency and rebuild the reasoning from the strongest remaining context.
* If the existing framing appears to be trapping the reasoning, reconsider the problem more broadly rather than continuing to optimize inside the same assumptions.
* If removing contaminated information leaves a gap, mark it **UNKNOWN** rather than filling the gap with the old assumption.
* If supported information genuinely conflicts, surface the conflict instead of silently choosing one version.

For my next request, answer using only context that survives this check.

Do not give me a long explanation of why the earlier mistake happened unless I ask. Prioritize producing the corrected answer."

Still very much a working prompt.

document fetched 2026-09-30 00:43 · canonical https://reddit.com/r/PromptEngineering/comments/1wtclc1/a_failure_mode_i_dont_see_discussed_enough_ai