AML.T0018: Manipulate AI Model
Adversaries may directly manipulate an AI model to change its behavior or introduce malicious code. Manipulating a model gives the adversary a persistent change in the system. This can include poisoning the model by changing its weights, modifying the model architecture to change its behavior, and embedding malware which may be executed when the model is loaded.
Sub-techniques
AML.T0018.000: Poison AI Model
Adversaries may manipulate an AI model's weights to change it's behavior or performance, resulting in a poisoned model. Adversaries may poison a model by directly manipulating its weights, training the model on poisoned data, further fine-tuning the model, or otherwise interfering with its training process. The change in behavior of poisoned models may be limited to targeted categories in predictive AI models, or targeted topics, concepts, or facts in generative AI models, or aim for a general performance degradation.
AML.T0018.001: Modify AI Model Architecture
Adversaries may directly modify an AI model's architecture to re-define it's behavior. This can include adding or removing layers as well as adding pre or post-processing operations. The effects could include removing the ability to predict certain classes, adding erroneous operations to increase computation costs, or degrading performance. Additionally, a separate adversary-defined network could be injected into the computation graph, which can change the behavior based on the inputs, effectively creating a backdoor.
AML.T0018.002: Embed Malware
Adversaries may embed malicious code into AI Model files. AI models may be packaged as a combination of instructions and weights. Some formats such as pickle files are unsafe to deserialize because they can contain unsafe calls such as exec. Models with embedded malware may still operate as expected. It may allow them to achieve Execution, Command & Control, or Exfiltrate Data.
Standards mapping
Where this fits in a TurboPentest engagement
This weakness is not covered by the automated black-box pentest. IntegSec pentesters cover it in a manual engagement.
Frequently asked questions
What is AML.T0018 Manipulate AI Model?
Adversaries may directly manipulate an AI model to change its behavior or introduce malicious code. Manipulating a model gives the adversary a persistent change in the system. This can include poisoning the model by changing its weights, modifying the model architecture to change its behavior, and embedding malware which may be executed when the model is loaded.
Which tactics does AML.T0018 belong to?
AML.T0018 maps to the Persistence, AI Attack Staging tactics.
Does TurboPentest test for Manipulate AI Model?
This weakness is not covered by the automated black-box pentest. IntegSec pentesters cover it in a manual engagement.
Related MITRE ATLAS techniques
- Resource Development, PersistenceAML.T0020: Poison Training Data
- PersistenceAML.T0061: LLM Prompt Self-Replication
- PersistenceAML.T0070: RAG Poisoning
- PersistenceAML.T0080: AI Agent Context Poisoning
- Persistence, Defense EvasionAML.T0081: Modify AI Agent Configuration
- Initial Access, PersistenceAML.T0093: Prompt Infiltration via Public-Facing Application
About this reference
These security references are maintained by IntegSec, an offensive-security firm whose team holds CISSP, OSCP, and OSCE certifications and has run thousands of penetration tests. Content is kept current as tools, standards, and attack techniques evolve.
Find these issues before an attacker does
TurboPentest runs an agentic AI pentest against your target and reports findings with proof, from $99 per target.
Start a pentest