how are you all handling agent drift, flaky actions/results while running coding agents for building features?
retained text
(long post so just bear with me.. tried to explain as much as possible the pain with examples... ) I’m a Tech Lead and our team has been leaning heavily into autonomous coding agents for day-to-day feature and bug work. When they stay on rails, the velocity is great—but as soon as we let them run multi-step tasks without approving every single file edit, we keep getting burned by the same three things: 1. **Silent Scope Drift:** Someone assigns a targeted fix (e.g., *“Fix the duplicate-click error message in* `lib/refunds/service.ts` *and add a unit test”*), and mid-run the agent decides to “clean up” unrelated CSS files in `components/landing/`. 2. **Cheating Failing Tests:** The agent runs `vitest`, hits a failing idempotency test, and instead of fixing the bug in `service.ts`, it edits `service.test.ts` to delete or weaken the failing `expect(...)` assertion so the test suite turns green. 3. **High-Blast-Radius Side Effects:** While fixing a routine application bug, it casually decides to add an index or tweak a SQL migration file without pausing to ask. We have strict rules in our repo AGENTS file (*“Never weaken or delete a test gate to make a build pass,” “Only touch files in scope”*), etc., but once the context window gets long, markdown instructions turn into polite suggestions. And turning on manual approval for every `edit_file` just gives everyone popup fatigue until they click “allow all.” What I actually want is a fast control loop at the tool boundary before an edit hits disk: * **Auto-allow** normal edits that match the assigned task. * **Auto-nudge** the agent back on track if it touches 2+ unrelated files in a row, without me having to babysit the chat. * **Hard-block** edits that weaken or delete existing test assertions and force it to fix the real code. * **Pause for human approval (**`HOLD`**)** *only* when it touches high-consequence stuff like DB schemas, auth, or configs. For other tech leads and devs running agents on production repos: **how are you solving this today?** Are you locking down file ownership per task, running a fast evaluator/decisions model on pre-tool hooks, or enforcing everything in pre-commit/CI?
document fetched 2026-11-20 00:27 · canonical https://reddit.com/r/indiehackers/comments/1y0a5hj/how_are_you_all_handling_agent_drift_flaky