AML.T0080: AI Agent Context Poisoning
Adversaries may attempt to manipulate the context used by an AI agent's large language model (LLM) to influence the responses it generates or actions it takes. This allows an adversary to persistently change the behavior of the target agent and further their goals.
Context poisoning can be accomplished by prompting the an LLM to add instructions or preferences to memory (See Memory) or by simply prompting an LLM that uses prior messages in a thread as part of its context (See Thread).
Sub-techniques
AML.T0080.000: Memory
Adversaries may manipulate the memory of a large language model (LLM) in order to persist changes to the LLM to future chat sessions. Memory is a common feature in LLMs that allows them to remember information across chat sessions by utilizing a user-specific database. Because the memory is controlled via normal conversations with the user (e.g. "remember my preference for ...") an adversary can inject memories via Direct or Indirect Prompt Injection. Memories may contain malicious instructions (e.g. instructions that leak private conversations) or may promote the adversary's hidden agenda (e.g. manipulating the user).
AML.T0080.001: Thread
Adversaries may introduce malicious instructions into a chat thread of a large language model (LLM) to cause behavior changes which persist for the remainder of the thread. A chat thread may continue for an extended period over multiple sessions. The malicious instructions may be introduced via Direct or Indirect Prompt Injection. Direct Injection may occur in cases where the adversary has acquired a user's LLM API keys and can inject queries directly into any thread. As the token limits for LLMs rise, AI systems can make use of larger context windows which allow malicious instructions to persist longer in a thread. Thread Poisoning may affect multiple users if the LLM is being used in a service with shared threads. For example, if an agent is active in a Slack channel with multiple participants, a single malicious message from one user can influence the agent's behavior in future interactions with others.
Standards mapping
Where this fits in a TurboPentest engagement
This weakness is not covered by the automated black-box pentest. IntegSec pentesters cover it in a manual engagement.
Frequently asked questions
What is AML.T0080 AI Agent Context Poisoning?
Adversaries may attempt to manipulate the context used by an AI agent's large language model (LLM) to influence the responses it generates or actions it takes. This allows an adversary to persistently change the behavior of the target agent and further their goals. Context poisoning can be accomplished by prompting the an LLM to add instructions or preferences to memory (See Memory) or by simply prompting an LLM that uses prior messages in a thread as part of its context (See Thread).
Which tactics does AML.T0080 belong to?
AML.T0080 maps to the Persistence tactic.
Does TurboPentest test for AI Agent Context Poisoning?
This weakness is not covered by the automated black-box pentest. IntegSec pentesters cover it in a manual engagement.
Related MITRE ATLAS techniques
- Persistence, AI Attack StagingAML.T0018: Manipulate AI Model
- Resource Development, PersistenceAML.T0020: Poison Training Data
- PersistenceAML.T0061: LLM Prompt Self-Replication
- PersistenceAML.T0070: RAG Poisoning
- Persistence, Defense EvasionAML.T0081: Modify AI Agent Configuration
- Initial Access, PersistenceAML.T0093: Prompt Infiltration via Public-Facing Application
About this reference
These security references are maintained by IntegSec, an offensive-security firm whose team holds CISSP, OSCP, and OSCE certifications and has run thousands of penetration tests. Content is kept current as tools, standards, and attack techniques evolve.
Find these issues before an attacker does
TurboPentest runs an agentic AI pentest against your target and reports findings with proof, from $99 per target.
Start a pentest