← sources

Stop checking what your agent said — check what it changed

source: reddit (reddit:AI_Agents)published 2026-10-25 06:08ingested 2026-10-26 02:35

retained text

Most agent verification setups check the wrong thing: they re-read the agent's own transcript and ask whether the story sounds consistent. A transcript can be perfectly consistent and still wrong — the agent can describe a change it never made.

Three patterns that actually catch mistakes, drawn from what keeps coming up in verification discussions:

1. The verifier must not trust the actor's path. An independent check reads state through its own read path — the database, the API, the destination system — not the agent's summary of what it did. If the checker and the actor share the same view, you've built a second opinion from the same source.

2. 'Deferred' needs a shape, not just a label. A verification state that just says 'open' or 'watchlist' rots: nobody owns it and nothing forces it closed. I'd require every deferred item to name the exact proof that's missing and what re-triggers it — a reappearing record, a deadline, a human review. An open item without a trigger is a graveyard.

3. Readbacks have to split render from data. Reloading the page and seeing the field is a UI assertion, not a persistence proof — the field can render from cache or a lagging replica while the write never landed. I'd assert the visible value and, separately, the stored value through an authoritative read. One without the other is half a check.

The common thread: verification is about the effect on the world, not the quality of the story about it.

Where do your verifiers still share a path with the actor — and have you caught a case where that shared path hid a real miss?

document fetched 2026-10-26 00:19 · canonical https://reddit.com/r/AI_Agents/comments/1xeumsr/stop_checking_what_your_agent_said_check_what_it