AML.T0043: Craft Adversarial Data
Adversarial data are inputs to an AI model that have been modified such that they cause the adversary's desired effect in the target model. Effects can range from misclassification, to missed detections, to maximizing energy consumption. Typically, the modification is constrained in magnitude or location so that a human still perceives the data as if it were unmodified, but human perceptibility may not always be a concern depending on the adversary's intended effect. For example, an adversarial input for an image classification task is an image the AI model would misclassify, but a human would still recognize as containing the correct class.
Depending on the adversary's knowledge of and access to the target model, the adversary may use different classes of algorithms to develop the adversarial example such as White-Box Optimization, Black-Box Optimization, Black-Box Transfer, or Manual Modification.
The adversary may Verify Attack their approach works if they have white-box or inference API access to the model. This allows the adversary to gain confidence their attack is effective "live" environment where their attack may be noticed. They can then use the attack at a later time to accomplish their goals. An adversary may optimize adversarial examples for Evade AI Model, or to Erode AI Model Integrity.
Sub-techniques
AML.T0043.000: White-Box Optimization
In White-Box Optimization, the adversary has full access to the target model and optimizes the adversarial example directly. Adversarial examples trained in this manner are most effective against the target model.
AML.T0043.001: Black-Box Optimization
In Black-Box attacks, the adversary has black-box (i.e. AI Model Inference API Access via API access) access to the target model. With black-box attacks, the adversary may be using an API that the victim is monitoring. These attacks are generally less effective and require more inferences than White-Box Optimization attacks, but they require much less access.
AML.T0043.002: Black-Box Transfer
In Black-Box Transfer attacks, the adversary uses one or more proxy models (trained via Create Proxy AI Model or Train Proxy via Replication) they have full access to and are representative of the target model. The adversary uses White-Box Optimization on the proxy models to generate adversarial examples. If the set of proxy models are close enough to the target model, the adversarial example should generalize from one to another. This means that an attack that works for the proxy models will likely then work for the target model. If the adversary has AI Model Inference API Access, they may use Verify Attack to confirm the attack is working and incorporate that information into their training process.
AML.T0043.003: Manual Modification
Adversaries may manually modify the input data to craft adversarial data. They may use their knowledge of the target model to modify parts of the data they suspect helps the model in performing its task. The adversary may use trial and error until they are able to verify they have a working adversarial input.
AML.T0043.004: Insert Backdoor Trigger
The adversary may add a perceptual trigger into inference data. The trigger may be imperceptible or non-obvious to humans. This technique is used in conjunction with Poison AI Model and allows the adversary to produce their desired effect in the target model.
Standards mapping
Where this fits in a TurboPentest engagement
This weakness is not covered by the automated black-box pentest. IntegSec pentesters cover it in a manual engagement.
Frequently asked questions
What is AML.T0043 Craft Adversarial Data?
Adversarial data are inputs to an AI model that have been modified such that they cause the adversary's desired effect in the target model. Effects can range from misclassification, to missed detections, to maximizing energy consumption. Typically, the modification is constrained in magnitude or location so that a human still perceives the data as if it were unmodified, but human perceptibility may not always be a concern depending on the adversary's intended effect. For example, an adversarial input for an image classification task is an image the AI model would misclassify, but a human would still recognize as containing the correct class. Depending on the adversary's knowledge of and access to the target model, the adversary may use different classes of algorithms to develop the adversarial example such as White-Box Optimization, Black-Box Optimization, Black-Box Transfer, or Manual Modification. The adversary may Verify Attack their approach works if they have white-box or inference API access to the model. This allows the adversary to gain confidence their attack is effective "live" environment where their attack may be noticed. They can then use the attack at a later time to accomplish their goals. An adversary may optimize adversarial examples for Evade AI Model, or to Erode AI Model Integrity.
Which tactics does AML.T0043 belong to?
AML.T0043 maps to the AI Attack Staging tactic.
Does TurboPentest test for Craft Adversarial Data?
This weakness is not covered by the automated black-box pentest. IntegSec pentesters cover it in a manual engagement.
Related MITRE ATLAS techniques
About this reference
These security references are maintained by IntegSec, an offensive-security firm whose team holds CISSP, OSCP, and OSCE certifications and has run thousands of penetration tests. Content is kept current as tools, standards, and attack techniques evolve.
Find these issues before an attacker does
TurboPentest runs an agentic AI pentest against your target and reports findings with proof, from $99 per target.
Start a pentest