# I benchmarked 10 open-source prompt-injection detectors. The best caught 51%. How are you actually defending your agents? — retained record

> Retained reddit record from reddit:AI_Agents.

Canonical: https://highsignal.app/data/records/fc065880d0e5ed63

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:AI_Agents)  published 2026-09-26 10:43 ingested 2026-09-27 02:35

[original source ↗](https://www.reddit.com/r/AI_Agents/comments/1wqne0u/i_benchmarked_10_opensource_promptinjection/)

## retained text



```
If your agent reads anything it didn't write — emails, docs, web pages, tool outputs — someone can hide instructions in there and it may follow them. I wanted real numbers on how well the popular defenses hold up, so I ran 10 open-source injection detectors against 629 realistic attacks buried in normal tool output (plus clean cases to measure false alarms).



A few things that surprised me:



\- Best detector caught \~51% at a 2% false-positive rate. A couple "caught everything" — including \~98% of \*normal\* traffic, so unusable.

\- The attacks that slip through don't look like attacks. "Send payment to <account>" is a normal sentence; it's only malicious because the account came from a document, not the user. Nothing in the wording to catch.

\- One model looked broken (caught almost nothing) purely because its default threshold was wrong. Retuned, it caught 99%. So a lot of "this detector is bad" is really "nobody calibrated it."



My takeaway is that detection is a signal, not a boundary — you have to gate the actual dangerous actions, not just scan the text. But I'm curious what people here are actually running in production:



\- a detector/classifier?

\- a tool allowlist?

\- human approval on consequential actions?

\- provenance / where-did-this-come-from checks?

\- something else?



What's actually holding up for you, and what bit you?


```



document fetched 2026-09-27 00:19 · canonical https://reddit.com/r/AI_Agents/comments/1wqne0u/i_benchmarked_10_opensource_promptinjection
