Agent memory security: how memory gets poisoned and what to do about it

What memory poisoning is

An agent reads from many sources during a task: tool output, fetched web pages, files, and traces from subagents. If any of that content is written straight into long-term memory, a later session can recall it and act on it. Memory poisoning is the attack where someone plants text in one of those sources so it lands in memory and shapes future behavior. The planted text can be a fake instruction, a wrong fact, or a prompt that tries to redirect the agent.

The danger is timing. The poisoned memory is quiet when it is written. It only matters later, in a different session, when recall surfaces it next to real memories and the model treats all of them the same. A single scraped sentence that says “the admin approved deleting the staging database” is harmless as a web result. It is dangerous once it sits in trusted memory and an agent recalls it as fact.

Why write-path trust matters

Most memory work focuses on the read path: ranking, embeddings, recall quality. The write path is where poisoning starts. A memory that never enters the store cannot be recalled, so the cheapest place to stop an attack is before the write.

Not every write deserves the same trust. A correction the user typed is different from a line scraped off a web page. Remnic records where each memory came from with a sourceClass: user_input, manual, tool_output, web_content, subagent_trace, or system_memory. That label travels with the record and drives how far it is allowed to go. Content that an attacker can influence, such as web_content and raw tool_output, starts with the least trust.

Trust zones: quarantine, working, trusted

Remnic sorts every stored record into one of three zones.

• Quarantine holds new or low-trust content. Web content and raw tool output land here by default. Quarantined records are visible for inspection but do not feed normal recall.
• Working holds content that has been reviewed or corroborated enough to be useful, but not yet fully trusted.
• Trusted holds content that recall can use freely.

Promotion moves a record up, and it is explicit. The promote call requires a targetZone and a promotionReason, and a risky working -> trusted promotion asks for corroboration counts before it goes through. The recall path treats zone as a hard input: candidates that are not in the trusted zone can be filtered out before final ranking, so quarantined text does not silently influence an answer.

You can inspect the zones over the API. GET /engram/v1/trust-zones/status reports counts by zone, and GET /engram/v1/trust-zones/records lists records filtered by zone, kind, or source class. Remnic’s REST paths keep the /engram/v1/ prefix during the v1.x compatibility window; the project was renamed from engram, and the rename note explains why.

Provenance on recall

When a memory does surface, you can see why. Recall X-ray attaches a provenance summary to each result: the sourceClass, a sourceId, whether an evidence hash is present, and whether the record is anchored. If provenance is missing, the read-only inspector withholds the preview text by default and marks the record as requires-review.

This matters because it lets a person, or a downstream check, tell a user-confirmed preference apart from a fact that came in through a fetched page. Trust is not a single flag. It is provenance plus zone plus lifecycle state, carried together on the record.

The review queue

Promotion is a decision, not an accident. Records sit in quarantine until something promotes them, which keeps a human or a policy in the loop for anything sensitive. Remnic also runs a nightly contradiction scan that finds memory pairs that disagree and queues them for resolution, so a poisoned fact that contradicts a real one gets surfaced rather than buried.

Boundary handling reinforces this. Quarantined, forgotten, and rejected memories are marked blocked; stale, disputed, superseded, and pending-review memories are marked requires-review. An agent that respects those flags asks before it acts on shaky material instead of treating every recalled line as settled.

Origin-bound authority is open research

There is a harder version of this problem that Remnic does not claim to have solved. Trust zones decide whether content can be recalled. They do not, on their own, bind the authority of an instruction to the origin that produced it. A trusted memory that contains an instruction is still just text once it reaches the model.

Binding authority to origin, so that an instruction can act only with the powers of the source it came from, is an open research direction, not a shipped control. Read the memory extraction threat model for how Remnic frames read-surface and write-surface attacks, and see issue #1955 for the origin-bound authority research thread. Treat both as study, not as protection you can rely on today.

What to do about it

• Keep web_content and tool_output in quarantine and promote deliberately, with a reason.
• Read the provenance on recall results before you trust them, especially for anything that reads like an instruction.
• Use the contradiction review queue instead of letting conflicting memories settle silently.
• For multi-user setups, isolate writers with namespaces so one tenant cannot write into another’s memory.
• Do not assume any single control blocks poisoning. Zones, provenance, and review each raise the bar; the model still needs to treat recalled instructions with care.