← sources

how to stop agents from screwing up?

source: reddit (reddit:AI_Agents)published 2026-09-29 09:38ingested 2026-09-30 02:37

retained text

i'm a final-year CS student and I'm scoping my capstone around agent safety, so I'm trying to talk to people who've actually shipped agents that take real actions rather than just chat.

bit of background on why this is personal. Over the summer I interned at a startup and built an agent that scraped funding-round news and matched new rounds to our ideal customer profile, basically to feed our outbound. The scary part wasn't the model being dumb, it was the model being confident and wrong while having access to real systems. I ended up hand-writing a pile of checks so it wouldn't \[write junk into our CRM / act on a half-scraped page / loop and burn through calls\]. It worked, but it felt fragile and totally ad hoc, and I basically rebuilt the same safety net from scratch every time.

i've looked at what's out there, Brane Core, some of the guardrail SDKs, Promptfoo, and most of them either check one action at a time or focus on prompt-level stuff, which doesn't fully match what actually bit me. So before I commit to a direction I want to hear how people really handle this.

i am curious about:

* What's the worst thing an agent of yours has done, or nearly done, when it had real tool access? What was the actual failure?
* When it goes wrong, is it usually one bad action, or a chain of individually-fine steps that added up to something bad?
* What do you actually do about it today, roll your own checks, use a tool, human-in-the-loop, or just watch it?
* If you've tried existing guardrail tools and dropped them, what made you drop them?

i am just trying to figure out if this is a real enough problem to build my capstone on.

document fetched 2026-09-30 00:19 · canonical https://reddit.com/r/AI_Agents/comments/1wt5p5q/how_to_stop_agents_from_screwing_up