This survey examines a recent class of adversarial efficiency degradation attacks that target these mechanisms to increase computation without necessarily degrading accuracy, and unify and compare two representative attacks across three popular token-pruning frameworks.
Abstract
Vision Transformers (ViTs) increasingly rely on input-adaptive inference, such as token pruning and early halting, to meet energy and latency budgets. This survey examines a recent class of adversarial efficiency degradation attacks that target these mechanisms to increase computation without necessarily degrading accuracy. We unify and compare two representative attacks, SlowFormer (a universal adversarial patch) and DeSparsify (per-image perturbations), across three popular token-pruning frameworks: A-ViT, ATS, and AdaViT. We standardize reporting using GFLOPs, accuracy loss, and an Attack Success (AS) metric that measures how much of the model's compute savings the attack takes away. Understanding these attacks is crucial for designing countermeasures that not only mitigate risk but also remain lightweight, since deployment often occurs in low-power settings such as mobile or embedded devices. To organize our analysis, we focus on three questions: how input-adaptive optimizations (e.g., token pruning and early halting) create attack surfaces for efficiency degradation; how such attacks operate in practice and which optimizations are most vulnerable; and which defenses exist today and whether they meaningfully restore efficiency under attack.
MOAT is proposed, a model-agnostic pre-processing defense pipeline that applies a combination of input transformations to protect efficient ViT implementations against adversarial efficiency attacks.
Anadi Goyal, Nandish Chattopadhyay, C. Karfa et al.· 0 citations
This work comprehensively investigates computation-efficient strategies to speed up latent adversarial training from two complementary perspectives, and reduces per-step adversarial-training FLOPs by 48.1% while requiring only 0.0118% trainable parameters.
This study reveals an Asymmetric Adversarial Trajectory (AAT) property in LIC systems: transitioning from adversarial to benign regions is significantly easier than the reverse process, where adversarial examples can often be roughly recovered within only 1-2 steps.
Dual Modular Redundancy (DMR) and Triple Modular Redundancy (TMR) are commonly used methods for providing fault detection and/or tolerance in safety-critical systems by incorporating redundant – and often diverse – components. However, these systems can still be susceptible to adversarial attacks that may deceive AI models, potentially leading to severe consequences. In this paper, we introduce enhanced DMR and TMR strategies for image-based object detection, leveraging image transformations during inference to help reduce the impact of adversarial inputs, while preserving the inherent advantages of diverse redundancy for safety purposes. Experimental results demonstrate that our approach significantly improves robustness under adversarial conditions, achieving up to 12.9% and 12.2% higher accuracy than state-of-the-art solutions in DMR and TMR configurations, respectively, when attacks are individually crafted for each image. Furthermore, against universal adversarial attacks, our solution achieves even greater accuracy gains, with up to 26.8% and 26.0% higher accuracy in DMR and TMR configurations, respectively.
Martí Caro, Axel Brando, Jaume Abella· ACM Transactions on Design A...· 0 citations
This paper introduces the first attack that directly optimizes an encoder-attention objective under an imperceptible, bounded, bounded perturbation, and argues that encoder attention concentrates the model's spatial reasoning, so corrupting it propagates through the detection pipeline more disruptively than perturbing the detection output alone.
Ridma Jayasundara, Shaheer Mohamed, Tharindu Fernando et al.· 0 citations
This work establishes a high-probability generalization bound for ViTs in classification tasks under adversarial settings, and elucidates the roles of several factors in mitigating perturbation effects, norm regularization of weight matrices and depth-wise propagation constraints on layer-wise norms.
Zi-Wen Jiang, Chang Cao, Han Li et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
