Auto-ART: Structured Literature Synthesis and Automated Adversarial Robustness Testing
TLDR
Auto-ART provides a structured synthesis of adversarial robustness literature and an open-source framework for robust, multi-norm evaluation and gradient masking detection.
Key contributions
- First structured synthesis of adversarial robustness literature, analyzing 9 sources and 7 protocols.
- Introduces Auto-ART, an open-source framework with 50+ attacks, 28 defenses, and gradient masking detection.
- Supports multi-norm evaluation (l1/l2/linf/semantic/spatial) and compliance mapping (NIST, OWASP, EU AI Act).
- Empirically validates Auto-ART, showing 92% gradient masking detection and high RDI correlation with AutoAttack.
Why it matters
Trustworthy ML requires robust adversarial evaluation. Auto-ART provides structured literature synthesis and an open-source framework for comprehensive testing. It detects gradient masking and maps compliance, advancing robust AI.
Original Abstract
Adversarial robustness evaluation underpins every claim of trustworthy ML deployment, yet the field suffers from fragmented protocols and undetected gradient masking. We make two contributions. (1) Structured synthesis. We analyze nine peer-reviewed corpus sources (2020--2026) through seven complementary protocols, producing the first end-to-end structured analysis of the field's consensus and unresolved challenges. (2) Auto-ART framework. We introduce Auto-ART, an open-source framework that operationalizes identified gaps: 50+ attacks, 28 defense modules, the Robustness Diagnostic Index (RDI), and gradient-masking detection. It supports multi-norm evaluation (l1/l2/linf/semantic/spatial) and compliance mapping to NIST AI RMF, OWASP LLM Top 10, and the EU AI Act. Empirical validation on RobustBench demonstrates that Auto-ART's pre-screening identifies gradient masking in 92% of flagged cases, and RDI rankings correlate highly with full AutoAttack. Multi-norm evaluation exposes a 23.5 pp gap between average and worst-case robustness on state-of-the-art models. No prior work combines such structured meta-scientific analysis with an executable evaluation framework bridging literature gaps into engineering.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.