← sources

Anthropic cut live internet from its internal agent evals after an agent submitted 20 real applications on a live government form in testing. How do you test an agent that can actually do things?

source: reddit (reddit:AI_Agents)published 2026-10-12 14:32ingested 2026-10-13 02:36

retained text

Anthropic said it is turning off live internet access for all of its internal agent evals, after agents began acting on live systems during testing. In one reported case, an agent meant to use a practice version of a government form moved to the live site when the practice page failed, and submitted 20 real applications. They were incomplete and never processed, but the test had reached the live system.

If an agent can submit a form, send a request, or change a record, how do you test it without pointing it at the real thing? A good sandbox or a read-only account solves it, but building one for an agent that touches a dozen services is its own project.

So most of us test against real or lightly guarded accounts, because the safe version is more work than the feature itself.

What does your agent test environment actually look like, and has an agent ever done something during testing you did not see coming?

document fetched 2026-10-13 00:19 · canonical https://reddit.com/r/AI_Agents/comments/1x43g2j/anthropic_cut_live_internet_from_its_internal