The compounding error problem: how do you decide which steps get autonomy and which stay hard-coded?
retained text
Something I keep running into when designing agents: per-step reliability compounds, and it compounds fast. **The math that bugs me** If the model picks the right action 95% of the time (generous for open-ended tool use): - 5 steps: ~77% end-to-end - 10 steps: ~60% - 20 steps: ~36% And the failures are rarely clean crashes. It's the agent confidently taking a wrong turn that you only notice three steps later. **Where I've landed so far** Code owns the control flow, the model owns the judgment calls: - the LLM handles the fuzzy parts (classify, extract, decide between 2-3 options, draft) - every output gets validated by plain code before the next step - the model only gets to pick its own next step where the path genuinely can't be known in advance My rough test: if I can draw the flowchart on a whiteboard, it probably shouldn't be fully agentic. **Where full autonomy seems to work** 1. The path really is unknown up front 2. A human reviews the result anyway 3. A wrong turn is cheap to undo Coding agents tick all three (diff review, tests, git revert), which is probably why they're the clearest success story. **What I'm curious about** - Has anyone here got a fully autonomous agent in prod, no human in the loop, choosing its own steps? What does it do? - How are you measuring per-step reliability, and what actually moved it up? Better tool descriptions, evals, smaller action spaces, retries with verification? - Did anyone go the other way: start with a rigid workflow and loosen it into an agent once the data showed it was safe?
document fetched 2026-10-29 00:19 · canonical https://reddit.com/r/AI_Agents/comments/1xhnyk2/the_compounding_error_problem_how_do_you_decide