← sources

Question for people who actually run agents with human approval steps.

source: reddit (reddit:AI_Agents)published 2026-10-02 13:34ingested 2026-10-03 02:35

retained text

Most setups I've seen handle risky agent actions the same way: the agent asks, a human approves. But the data on that model isn't great.

A browser game built by Alex Wauters at ScaleX puts you in the approver's seat for an AI coding agent: 60 seconds to approve or deny its commands. After 40k+ plays and 409k decisions, players let about 1 in 3 malicious commands through. The most-missed one was approved ~65% of the time, even though the log right above the prompt showed what it did.

And before anyone says "it was a game": doctors, with their licenses on the line, override somewhere between half and almost all drug safety alerts in published studies. Stakes make people read. Volume makes them stop.

Try it yourself first, it's a humbling 60 seconds: "llmgame-scalex-dev" REPLACE THE "-" WITH "."

I'm trying to gauge whether a different model would be useful as an option in agent management systems. Roughly:

Blind second review: an AI reviewer independently judges each risky action. The human doesn't see its opinion first, so they don't just copy it.

Disagreement escalates: if the human approves something the AI flagged, a second person signs off before it runs.

Canaries: the TSA trick. Airport screeners get fake threats projected into real X-ray images to check they're paying attention. Same here: occasionally a synthetic "obviously deny this" action lands in the queue (logged, never executed) to see who's actually reading.

Earned approval authority: each reviewer builds an accuracy record. Miss too many canaries, or keep overriding flags that turn out bad, and your approval scope narrows until someone looks at it.

What I'd really like to know from people doing this day to day:

Is approval fatigue a real problem on your team, or do you keep prompts low enough that it never kicks in?

Would you turn this on? Which parts: all of it, just the second AI reviewer, none?

Canaries specifically: useful, or would your team feel spied on? (Phishing simulations went from resented to normal. Would this?)

What breaks? The AI reviewer will be wrong sometimes too, and scoring only misses probably trains everyone to deny everything.

What are you doing today that actually works?

Not selling anything here. Trying to figure out whether this is worth building as an option, or whether I'm solving a problem nobody has.

document fetched 2026-10-03 00:18 · canonical https://reddit.com/r/AI_Agents/comments/1wvtyrv/question_for_people_who_actually_run_agents_with