# Do your agent evals test access being revoked halfway through a conversation? — retained record

> Retained reddit record from reddit:AI_Agents.

Canonical: https://highsignal.app/data/records/ecffcffe750b8d23

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:AI_Agents)  published 2026-10-20 07:52 ingested 2026-10-21 02:35

[original source ↗](https://www.reddit.com/r/AI_Agents/comments/1xam18t/do_your_agent_evals_test_access_being_revoked/)

## retained text



```
One eval case I'd add for an agent working with permissioned data: revoke access halfway through the conversation.

Toy setup:

- A user can access Project A. The agent retrieves an incident note and produces a summary.
- Their project access is removed.
- In the same session, they ask for another detail from that note.

If the application's policy requires current access, what should happen?

A fresh retrieval might be blocked while the old snippet, a memory entry or a cached answer is still available to the model. I'd test those paths separately:

1. A follow-up in the existing conversation.
2. The same question from another account through a shared cache.
3. A run that starts before revocation but produces its answer afterward.

For each case, I'd record the applicable access policy, the evidence supplied and when authorization was checked. A refusal on a new search wouldn't establish that the existing session or cache also respects the change.

This is about preventing a later disclosure under that policy. It can't take back information the user already received.

Where would you enforce this in your stack: retrieval, context construction, the response boundary, or a combination?
```



document fetched 2026-10-21 00:19 · canonical https://reddit.com/r/AI_Agents/comments/1xam18t/do_your_agent_evals_test_access_being_revoked
