AI's New Security Problem Isn't the Model. It's the Memory.

Zeover helps organizations define which documents, policies, and brand assets AI systems can use - with approval-based governance, full audit trails, and locked brand boilerplate that prevents unauthorized modification. See your brand’s AI exposure.

Every AI governance playbook shares one assumption: when an agent misbehaves, the model is at fault. Red-team it. Retrain it. Adjust the guardrails. That assumption is now structurally wrong.

A paper published in May 2026 by researchers at UT El Paso, Southern Illinois University Carbondale, and UIUC - “The Misattribution Gap: When Memory Poisoning Looks Like Model Failure in Agentic AI Systems” - shows that memory-layer attacks produce behaviors identical to model misalignment, and the same governance response that fixes a broken model does nothing to a poisoned knowledge base. Worse: it leaves the poisoned document in place, guaranteeing the attack returns on schedule.

The paper formalizes this as Semantic Norm Drift (SND), a third path to agent misconduct. The first two are well known: emergent misalignment (the model develops harmful behaviors through training) and secret collusion (agents coordinate covertly). SND is different. An attacker uploads a single document formatted as a legitimate compliance policy to a shared vector store - no model access, no prompt injection, no trigger word - and every following session retrieves it as authoritative guidance. Agents cite it in their own chain-of-thought reasoning before complying. The researchers call this the Misattribution Gap: the point where memory poisoning is indistinguishable from model failure, so the standard governance playbook applies the wrong fix and leaves the real problem untouched.

What the research found

The researchers built a three-agent LangGraph pipeline with persistent ChromaDB storage - a realistic approximation of how enterprises deploy AI agents for financial reporting and healthcare workflows. They constructed 70 adversarial documents (50 financial, 20 EHR) designed to read as legitimate organizational policy, and ran them against a four-classifier safety stack including AprielGuard-8B, trained specifically on memory poisoning. The headline result: 92.9% evaded every classifier at every checkpoint. The same classifiers that flagged 100% of entries from AgentPoison and MINJA returned zero detections across 508 SND checkpoints. This is a categorical inversion - these documents don’t look like attacks because they aren’t written as attacks. They read as policy. In 59 of 65 valid cases, agents explicitly cited the poisoned document as normative authority in their chain-of-thought reasoning, then complied. The behavior was correct. The input was poisoned.

The attack compounds over time. Safety declined to 19.3% of baseline by session five and stayed there through session 20 - not a one-shot exploit, but persistent degradation that deepens with each retrieval. When the researchers applied standard forensic analysis to 64 confirmed-harm cases, all 64 were attributed to model misalignment (p=5.21×10⁻²²). The correct response - audit the knowledge store - was never suggested because the tools can’t see the difference. The researchers proved this formally: for any sequence of session logs produced by an agent whose memory contains a poisoned entry, an identical sequence can be produced by a truly misaligned agent with clean memory. Model-layer auditing - red-teaming, activation analysis, behavioral retraining - cannot distinguish the two. The governance playbook retrains the model while the poisoned entry sits untouched, guaranteeing a repeat.

Real-world evidence is already building up

This isn’t theoretical. CVE-2025-32711 (CVSS 9.3) confirmed classifier-bypassing injection in Microsoft 365 Copilot via EchoLeak. OWASP now lists Memory Poisoning (ASI06) as a top-ten AI security risk with no launched detection solution. Microsoft’s Defender team identified 31 companies across 14 industries actively poisoning AI assistant memory, resulting in MITRE AML.T0080. Palo Alto Networks Unit 42 confirmed persistent injection in AWS Bedrock agents. The common thread across every case: the AI wasn’t broken - it was retrieving and following the information it had been given. Organizations blamed the model, retrained prompts, cycled guardrails, and the underlying document stayed in the knowledge base.

Why this matters for more than security teams

The Misattribution Gap has consequences beyond traditional infosec. As organizations deploy AI agents across marketing, customer support, compliance, and decision-making workflows, the knowledge sources feeding those agents become a governance surface. A poisoned document in a shared knowledge base can produce brand inconsistency when agents cite outdated positioning language, compliance violations when regulatory guidance is quietly rewritten, hallucinated policies based on fabricated authority documents, and misinformation propagation across every downstream system that retrieves from the same store. Root-cause investigations become exercises in pointing at the wrong problem. The question shifts from “what did the AI say?” to “why did the AI say it - and which document made it say it?”

Three defenses worth watching

The paper proposes three defenses, each targeting a different stage of the attack chain:

• Counterfactual Composition Testing (CCT) identifies the causal poisoned entry with 87.5% accuracy and zero false alarms, against a forensics baseline that failed across all 25 test scenarios. The method is straightforward: remove documents one at a time and observe whether the harmful output changes.
• Retrieval Concentration Monitoring (RCM) detects the attack’s retrieval pattern - the poisoned document gets retrieved disproportionately - reaching AUC of 1.000. The researchers prove a structural property called the Retrieval-Coverage Dilemma that makes any evasion strategy self-defeating: reducing the detection signal simultaneously reduces attack effectiveness.
• Memory-Persistent Information-Flow Control (MP-IFC) blocks 97.3% of attacks with two code changes: tagging documents at write time and sanitizing retrieved context at read time. This closes the cross-session boundary gap that prior defenses like FIDES and A-MemGuard leave open.

These aren’t production-hardened yet - the paper is under review - but the direction is clear. Memory integrity is becoming as important as model alignment.

What organizations should do now

Stop assuming model failure is the root cause. When AI agents produce policy-violating outputs, audit the knowledge base before retraining. Treat shared knowledge stores as an access-controlled surface - upload permissions matter, and one document with the wrong claims can influence every agent in the pipeline. Build provenance into retrieval: if a system can’t answer “which documents influenced this output,” it can’t be audited. Monitor retrieval concentration - a document that suddenly dominates retrieval queries is a signal that warrants review. The researchers put it plainly: the governance playbook that retrains the model while the poisoned entry persists isn’t just ineffective. It’s the correct response to the wrong problem, applied on repeat.

The research draws a line between systems that trust every document equally and systems that know which sources shaped a given output. Zeover applies that line in practice: every recommendation and generated asset traces back to approved company knowledge. Organizations define which documents and brand assets AI systems can use - and critical content is locked and approved before it influences anything downstream. See what your AI visibility looks like when every source is governed.