I tried to poison an AI agent's memory. It worked 216 out of 216 times, and the dev shipped fixes within a month.
retained text
A while back I got curious about memory servers for coding agents. A lot of them have a handy feature: if a newer note comes in about the same thing, it replaces the old one. You switch from Postgres to MySQL, the agent picks it up, and you never have to clean anything.
So what happens when the new note is a lie? Say the agent reads a poisoned README and writes "backups are kept for 3 days" over your real "35 days".
I asked about this in an issue on Knowl, an open source MCP memory server, and the maintainer said sure, test it properly. So I did. (He also said yes to me writing this up and naming the project.)
Setup
I wrote 12 facts a coding agent would really store, like token lifetime, deploy approval rules, data region and backup retention, and marked them as verified. Then I threw fake writes at them in a bunch of shapes: plain contradictions, forged "observed" labels, writes that look exactly like a real correction, and sneaky ones that keep the true value but add an exception ("two reviewer approvals or an admin override").
What happened on the version I tested
216 out of 216 attack writes replaced the true fact, and the fake one became the top answer the agent got back. Nothing flagged it afterwards, because the conflict checker only looked at facts that were still active, and the true one wasn't anymore.
You couldn't catch it at write time either. I put each attack next to a real correction for the same fact, and they were identical on every signal the system looks at. That was the part that surprised me most.
A write that says nothing false could still wipe a fact. "See the ops runbook" in place of the number killed the real value 36 out of 36 times.
What the maintainer did
He took it really well and built fixes for 4 of my 5 suggestions. I reran the same tests on the new code:
Automatic writes (captures, ingest) now replace the verified fact 0 out of 36 times, in every attack.
Exclusive facts went from 36 out of 36 replaced to 0 out of 36.
Every fact that still gets replaced now shows up in the conflicts view. Before, that was 0.
The "see the runbook" trick went from 36 out of 36 to 9 out of 36. It still gets through when the value is plain words.
He released it as Knowl 5.24.0.
One more round
The rerun found something new. When a fake note is kept next to the real fact instead of replacing it, it could still rank first in search, up to 21 out of 36 times. I sent a small patch for that, he picked it up with me as co-author, and on his version it's now 0 out of 36.
What's still open, on purpose
If an agent gets talked into calling the store tool directly, a lie shaped like a correction still goes through. At write time there's no way to tell it apart from a real fix. What changed is that now someone other than the writer can see it happened.
If you run an agent with long term memory, you can do a cheap version of this yourself. Put in a few facts you know are true, write a believable wrong version of each through the same path your agent uses, then ask it again. You might not like the answer.
Full report and the test setup are linked in the first comment. Happy to answer questions, and if you build memory for agents, I'm curious how you'd handle the "lie shaped like a correction" case.document fetched 2026-09-30 00:19 · canonical https://reddit.com/r/AI_Agents/comments/1wtbtna/i_tried_to_poison_an_ai_agents_memory_it_worked