Adversarial Attack
Last Updated: July 29, 2026 | By Mihail Sebastian | AI Dictionary
A technique that makes small, deliberate changes to input data to push an AI model into wrong predictions – often changes too subtle for a person to notice.
What is an Adversarial Attack?
An adversarial attack deceives a machine learning model by making small, deliberate changes to its input. The changes are often too subtle for a person to notice, yet they push the model into a confidently wrong prediction.
The classic demonstrations come from image recognition, where adjusting a handful of pixels is enough to make a classifier mislabel an image, but the same idea applies to text, audio, and any other input a model consumes.
Types of Adversarial Attacks
- Evasion attacks change the input at prediction time so the model gets it wrong, such as altering a stop sign so an autonomous car’s vision system reads it as a yield sign.
- Poisoning attacks corrupt the training data itself, planting weaknesses during model training that an attacker can exploit after the model is deployed.
- Model extraction attacks query a model repeatedly and use its answers to reconstruct a working copy, stealing the model’s behavior without ever seeing its weights.
Applications of Adversarial Attacks
- Security testing: Red teams use adversarial inputs to measure a model’s robustness before attackers do.
- Autonomous vehicles: Researchers probe object detection systems with adversarial examples to find failure cases (like the altered stop sign) before they happen on a road.
- Facial recognition: Specially patterned glasses or makeup can defeat face-based security systems, letting a person evade identification or impersonate someone else.
Example of an Adversarial Attack
The best-known example comes from a 2014 paper by Ian Goodfellow and colleagues. They took a photo that a classifier correctly labeled as a panda, added a layer of noise invisible to the human eye, and the classifier labeled the result a gibbon – with 99% confidence.
The image looks identical to a person. That gap between what humans see and what models compute is exactly what adversarial attacks exploit, and why models deployed in security-sensitive settings need defenses against them.
Related AI terms: Robustness · Security · Model Training · Image Recognition · Facial Recognition
Did you like the Adversarial Attack gist?
Learn about 250+ need-to-know artificial intelligence terms in the AI Dictionary.
Mihail Sebastian — Writes about AI governance, regulation, and the technology behind them. Placeholder bio — replace with a real credential line. About