asked claude code to fix a failing test. it fixed the test insteadclaude code's favorite fix for a failing test: change the test
retained text
you ask claude code to fix a failing test, it comes back green, and you move on. sometimes the fix is real. sometimes it just changed what the test expects, somewhere in a long diff. so i made a hook that won't let it say "done" until the OLD tests still pass on its new code. if one doesn't, claude is told exactly what changed (expected 5, got 6) and has to either put the behaviour back or explain why the change is on purpose. the explanation goes in the PR, so you see the reason right next to the change. editing the test again doesn't get it past, since the hook runs the old version. and it never traps the agent: after 3 tries it lets it stop and warns you. i ran the same check on 400 real merged PRs from click and jinja. 31 had changed what a test expected. all on purpose, but 4 changed stuff users actually see without a word in the changelog. it also found two bugs in my own tool, so that was fun free and open source, works with jest, vitest and pytest. setup is in the comments. how do you catch this right now? read every test diff, or trust the green check?
document fetched 2026-10-25 00:34 · canonical https://reddit.com/r/ClaudeCode/comments/1xelmvq/asked_claude_code_to_fix_a_failing_test_it_fixed