MITRE ATLAS's own case-study library shows that the machine-learning models inside security products are not a hypothetical target: independent researchers and vendors' own research teams have already demonstrated adversarial attacks that evaded CylancePROTECT's AI antivirus engine, ProofPoint's email-filtering model, Kaspersky's antimalware classifier and a commercial phishing-detection system, and a real attacker poisoned the classification signal several vendors' engines used to label a ransomware family on VirusTotal. Anyone building or buying AI-assisted security tooling — including an on-device anomaly-detection model in an OT sensor, gateway or camera — needs to treat the detection model itself as part of the attack surface, not just a feature to advertise.
This guide draws directly on MITRE's own published ATLAS case-study dataset (fetched from the atlas-data GitHub repository), the primary write-up that dataset itself cites for the Cylance case, and NIST's and OWASP's own adversarial-ML publication pages — not a secondary summary.
Key takeaways
- Every case study below starts with the same ATLAS technique: AI-Enabled Product or Service (AML.T0047), filed under the AI Model Access tactic.
- Does the product's detection model ever return a raw or indirectly inferable confidence score to logs, headers or API responses outside the vendor's own trust boundary?
Why is an AI-based security product itself part of the attack surface?
Every case study below starts with the same ATLAS technique: AI-Enabled Product or Service (AML.T0047), filed under the AI Model Access tactic. ATLAS describes this as an adversary simply using "a product or service that uses artificial intelligence" to get access to the model behind it — and for a security product, that access requires nothing more exotic than buying or installing the product itself. A deployed malware classifier, spam filter or anomaly detector is, from this angle, a queryable system an attacker can probe for free, as many times as they like, with no need to breach anything first. This is the same category of risk the OWASP Machine Learning Security Top 10 catalogs as Input Manipulation and Model Theft, and the one place where it bites hardest is a product whose entire job is to say yes or no to a file, an email, or a URL.
Real cases: adversarial ML attacks that already bypassed security products
| ATLAS case study | Target | Type | What happened |
|---|---|---|---|
| AML.CS0003 Bypassing Cylance's AI Malware Detection | CylancePROTECT, Cylance Smart Antivirus | Exercise (Skylight Cyber) | Verbose logging exposed the model's reputation scoring; researchers found a secondary "override" model and crafted a near-universal string that, appended to known malware, flipped the verdict to benign. |
| AML.CS0008 ProofPoint Evasion (CVE-2019-20634) | ProofPoint Email Protection | Exercise | Per-word spam scores leaked in email headers let researchers train a working copy of the model, then craft malicious emails scored as safe by the live filter. |
| AML.CS0014 Confusing Antimalware Neural Networks | Kaspersky's antimalware ML model | Exercise (Kaspersky's own ML research team) | Gray-box access to only the client-side feature extractor, not the cloud model, was enough to evade detection on most adversarially modified malware files. |
| AML.CS0032 Attempted Evasion of ML Phishing Webpage Detection | A commercial ML phishing-website detector | Incident (observed in the wild) | Attackers manually altered brand logos (cropping, blurring, removal) to evade the visual-similarity classifier; the layered system's other components still caught most of the pages. |
| AML.CS0002 VirusTotal Poisoning | VirusTotal | Incident | A metamorphic-code tool generated "mutant" ransomware variants uploaded to the platform; several vendors' classifiers began labeling the mutants as the same ransomware family even though many would not execute. |
Two of these five are tagged exercise in ATLAS's own data — controlled demonstrations by independent researchers or, in Kaspersky's case, the vendor's own research team, run specifically to find the weakness before an attacker did. The other two, VirusTotal Poisoning and the phishing-detector evasion, are tagged incident: activity ATLAS records as having actually happened against a production system, discovered after the fact.
Which ATLAS tactics keep recurring across these cases?
A pattern holds across all four exercises: AI Model Access (buying or using the product) is followed by Discovery — specifically Discover AI Model Outputs (AML.T0063) — because both Cylance's verbose debug logs and ProofPoint's email-header scores leaked model output that was never meant to be externally visible. That leaked signal then feeds Resource Development, either by training a proxy model (Train Proxy via Replication, AML.T0005.001, in the ProofPoint case) or by reusing an existing tool built for the job (Adversarial AI Attack Implementations, AML.T0016.000, covers the metamorphic-code tool used against VirusTotal). The chain ends at Defense Evasion's Evade AI Model (AML.T0015), the step every one of these case studies shares. The practical lesson: the single highest-leverage fix across all four exercises would have been not leaking raw or inferable model-output scores to anything outside the vendor's own trust boundary in the first place.
What does this mean for OT and embedded ML in security products?
A growing number of OT and IoT security products ship their own on-device or edge ML model — an anomaly detector reading a PLC network tap, a camera's on-device threat classifier, a gateway's predictive-maintenance model repurposed to flag intrusions. These share the exact starting point the ATLAS case studies document: AI-Enabled Product or Service (AML.T0047) is trivially satisfied by anyone who can buy, install or physically reach the device, which describes most OT/IoT hardware far more easily than it describes a cloud-hosted email filter. And once a bypass against an embedded model is public, it typically cannot be patched as fast as a cloud-hosted one — the model ships inside firmware, on the same slow update cadence the rest of the device follows. Recording every embedded or edge model as a first-class inventory item, the same gap our guide to AI-BOM for embedded AI devices covers, is the prerequisite for even knowing a bypass needs patching.
Questions to ask before shipping or buying AI-assisted security tooling
Does the product's detection model ever return a raw or indirectly inferable confidence scoreto logs, headers or API responses outside the vendor's own trust boundary? Cylance Smart Antivirus and ProofPoint Email Protection both did, and that leak was the starting point of each bypass. Has the vendor's own team tested the model against adversarial input, the way Kaspersky's and (per ATLAS case study AML.CS0000) Palo Alto Networks' research teams tested their own production malware and command-and-control detectors before publishing the results? What happens when one layer of a multi-model detection pipeline is evaded?The phishing-detector incident (AML.CS0032) shows a layered system surviving a partial bypass — a single-model detector has no such fallback. For an embedded or OT deployment, how is a model bypass patched once it is public, and on what cadence relative to the device's other firmware updates?
SoftWSP's own AI-assisted reporting sits downstream of an evidence-based firmware and OT analysis pipeline, with an engineer reviewing every finding before it ships — it is not a live, internet-facing classifier making autonomous allow/block decisions the way the products in the case studies above are, so these specific evasion techniques do not transfer directly to it. What does transfer is the discipline: treat any AI-assisted detection feature, including your own, as a claim to test rather than a feature to assume. See Device & Firmware Security for how that threat-modeling extends to a product's own AI-assisted or on-device detection features alongside conventional firmware risk.
Case-study facts, targets, dates and ATLAS tactic/technique IDs above are drawn directly from MITRE's own published ATLAS dataset (the atlas-data GitHub repository) and, for the Cylance case, Skylight Cyber's own original write-up. OWASP and NIST facts are drawn from each project's own publication pages. This guide is general information, not security advice, and no case study above involves a SoftWSP product.
Sources
- MITRE ATLAS — official site (matrix, tactics, techniques, case studies)
- MITRE ATLAS Data — versioned case-study dataset (GitHub, includes AML.CS0002, AML.CS0003, AML.CS0008, AML.CS0014, AML.CS0032 and their tactic/technique mappings)
- Skylight Cyber: "Cylance, I Kill You!" (original write-up of the CylancePROTECT bypass, AML.CS0003)
- NIST AI 100-2e2025: Adversarial Machine Learning — Taxonomy and Terminology of Attacks and Mitigations (publication page)
- OWASP Machine Learning Security Top 10 project (official OWASP project page)
Frequently asked questions
What is MITRE ATLAS, and why does it matter for AI used inside a security product?
MITRE ATLAS (Adversarial Threat Landscape for AI Systems) is a public knowledge base of real adversary tactics, techniques and case studies targeting AI and machine-learning systems — see our guide to [MITRE ATLAS and its 16 tactics](/guides/mitre-atlas-ai-threat-matrix/) for the full framework. It matters specifically for security products because several of ATLAS's own documented case studies are not attacks on chatbots or image generators — they are attacks on the detection models inside commercial antivirus, email-filtering and anti-phishing products, the exact category of AI feature a security vendor is most likely to advertise as a strength.
Has an AI-based antivirus or email-security product actually been bypassed using adversarial machine learning?
Yes, more than once, and the write-ups are public. Per ATLAS case study AML.CS0003, researchers at Skylight Cyber reverse-engineered CylancePROTECT's reputation-scoring model and found a secondary "override" model whose positive verdicts beat the primary detector's negative one, letting them craft a near-universal string that made malicious files read as benign when appended to them. Per ATLAS case study AML.CS0008 (CVE-2019-20634), researchers found that ProofPoint's Email Protection system leaked per-word spam scores in email headers, used those scores to train a working copy of the model, and then crafted emails that scored as safe while still carrying a malicious payload. Per ATLAS case study AML.CS0014, Kaspersky's own ML research team showed that even gray-box access to just the feature-extraction step of their antimalware product — not the cloud model itself — was enough to evade detection on most adversarially modified malware files.
What is the difference between a MITRE ATLAS case study marked 'exercise' versus one marked 'incident'?
ATLAS's own dataset tags every case study with a case-study-type field of either exercise or incident. An exercise is a controlled demonstration, often run by independent researchers or by a vendor's own red team, that proves an attack is possible against a real production system without an actual malicious actor being involved — the Cylance (AML.CS0003), ProofPoint (AML.CS0008) and Kaspersky (AML.CS0014) cases above are all tagged exercise. An incident is attack activity ATLAS records as having actually happened in the wild: AML.CS0002 (VirusTotal Poisoning, where a metamorphic-code tool was used to flood the platform with ransomware variants that poisoned several vendors' classification signal for that family) and AML.CS0032 (a commercial ML phishing-detection system that attackers evaded in production by manually altering brand logos) are both tagged incident.
Does following OWASP or NIST's adversarial-ML guidance make an AI-assisted security product safe from these attacks?
No single checklist eliminates the risk; these references give a shared vocabulary and a starting set of categories to test against, not a certification. Our guides to the [OWASP Machine Learning Security Top 10](/guides/owasp-machine-learning-security-top-10/) and [NIST AI 100-2e2025's adversarial ML taxonomy](/guides/nist-ai-100-2e2025-adversarial-ml-taxonomy/) cover the same underlying attack classes — input manipulation, data poisoning, model extraction — that the MITRE ATLAS case studies above demonstrate against real products. Using all three together (ATLAS for adversary tactics and real precedent, OWASP and NIST for the vocabulary) is a more complete starting point than relying on any one of them alone, but testing a specific product's own detection model against adversarial input remains necessary — none of these references test your model for you.
In this cluster
Questions about this page?