What actually broke when we moved our agents from demo to production?Body:
retained text
Been building agentic workflows for clients for a while now, and the gap between “works in the demo” and “works on a Tuesday with real users” keeps surprising me. Some things that bit us: 1. **Tool calls fail quietly.** The agent gets an empty or half-broken response from an API and carries on anyway, confidently. We now check every tool output before the agent can reason over it. It’s boring but it saved us. 2. **Long chains drift.** Around step 6 or 7 the agent forgets what the user actually asked for. Restating the original goal at checkpoints helped more than any prompt tweak. 3. **Evals were an afterthought.** We tested the happy path and shipped. Now we keep a small set of real failure cases from production and rerun them on every change. It’s not fancy, but regressions get caught before users find them. 4. **Retrieval was the real problem, not the model.** Half our “hallucinations” were the agent faithfully answering from the wrong chunk. Fixing chunking and adding a relevance check did more than switching models. 5. **People want an undo button more than autonomy.** Letting users approve the risky steps built more trust than making the agent smarter. Curious what others hit. What’s the thing that only showed up once real users touched your agent?
document fetched 2026-10-22 00:19 · canonical https://reddit.com/r/AI_Agents/comments/1xbh4ln/what_actually_broke_when_we_moved_our_agents_from