Preprint

Corrupting Attention: Evasion-Based Adversarial Attacks on Encoder Attention in Detection Transformers

Ridma Jayasundara Shaheer Mohamed Tharindu Fernando Harshala Gammulle Basura Fernando Sanka Rasnayake A. Subramanyam S. Sridharan C. Fookes

Aug 2026 · 0 citations · 37 references

Computer Science

TL;DR

This paper introduces the first attack that directly optimizes an encoder-attention objective under an imperceptible, bounded, bounded perturbation, and argues that encoder attention concentrates the model's spatial reasoning, so corrupting it propagates through the detection pipeline more disruptively than perturbing the detection output alone.

Abstract

Adversarial vulnerabilities remain a major concern for the safe deployment of neural networks, particularly in object detection, a core task embedded in many safety-critical systems. Detection transformers have emerged as leading object detectors, yet their adversarial robustness remains comparatively underexplored. Most existing attacks target the detection output rather than the attention mechanism that makes these models distinctive. In this paper, we introduce the first attack that directly optimizes an encoder-attention objective under an imperceptible, bounded $\ell_\infty$ perturbation. Rather than introducing an attacker-owned sink token through a visible patch, it drives the model's own attention toward a corrupted target. We argue that encoder attention concentrates the model's spatial reasoning, so corrupting it propagates through the detection pipeline more disruptively than perturbing the detection output alone. Our attack reduces DETR-R50 mAP on COCO from 42.1 to 0.97, a $\sim 4\times$ reduction in resulting mAP over the strongest existing attack under an identical perturbation budget and iteration count. We further show that this vulnerability is not specific to a particular corruption objective: across four qualitatively distinct targets, dispersion, re-ranking, permutation, and peak-suppression, detection consistently drops below 3 mAP, suggesting that the weakness arises from disrupting the attention structure itself rather than from any single target. Finally, we demonstrate that the attack generalizes across attention formulations, reducing DINO-Swin-L from 56.8 to 1.44 mAP against 7.3 for the strongest prior attack, establishing state-of-the-art on both dense and deformable attention.

arXiv Semantic Scholar

View source

Similar papers

Preprint Aug 2026

Multi-Task Consistency-based Detection of Adversarial Attacks

This work proposes an efficient and effective adversarial attack detection scheme leveraging the multi-task perception within a complex vision system, and develops a consistency score metric to measure the inconsistency between vision tasks.

Cong Chen, J. Monteuuis, Jonathan Petit · 0 citations

From Detection to Deception: Are AI-Generated Image Detectors Adversarially Robust?

Yun-Yun Tsai, Rui-Ze Xu, Cheng-Zhi Mao et al. · 0 citations

Review Aug 2026

A Survey of Adversarial Efficiency Degradation for Vision Transformer by Exploiting Input-adaptive Optimization

This survey examines a recent class of adversarial efficiency degradation attacks that target these mechanisms to increase computation without necessarily degrading accuracy, and unify and compare two representative attacks across three popular token-pruning frameworks.

Anadi Goyal, Nandish Chattopadhyay, Anupam Chattopadhyay et al. · 0 citations

Understanding and Exploiting Phase Sensitivity for Attacking Large Vision–Language Models

This paper proposes a novel LVLM attack method, called BadPhase with further backdoor designs, to implant adversarial phase as triggers into any image inputs via data poisoning so as to control the LVLMs’ predictions and finds that LVLMs are sensitive to the phase-aware image structure.

Daizong Liu, Junhao Dong, Xiang Fang et al. · 0 citations

Preprint Aug 2026

Adversarial Robustness in Smishing Detection: A Comparative Analysis of Adversarial Fragility in Classical vs. Transformer-Based Detection Systems

It is demonstrated that clean-text performance is not a reliable predictor of adversarial robustness, and the results underscore the necessity for architecture-specific defences and frame smishing detection as an adversarial cybersecurity challenge rather than a static classification task.

Denzel Chiuseni, A. Bahizire, Silva Hama et al. · 0 citations

Preprint Sep 2026

CrACK: Adversarial Attacks on Cross-Model Consistency in Collaborative Vision Foundation Models

Training-free collaborative pipelines that integrate Vision Foundation Models such as CLIP, SAM, and DINO achieve strong open-vocabulary dense prediction and are increasingly deployed in safety-critical applications. The security of these systems is commonly assumed to follow from the robustness of their individual models. We challenge this assumption. We identify a vulnerability shared by every collaborative pipeline: each model consumes the intermediate output of another without verifying semantic consistency, an unverified premise that we term the semantic-spatial alignment dependency. Existing adversarial attacks target a single model and overlook this premise, leaving the inter-model interface entirely unguarded. We propose CrACK (Cross-model Adversarial Consistency attack), an inference-time attack that exploits this interface without modifying any input pixel, model weight, or training data. CrACK operates in two stages: Adversarial Affinity Contradiction Injection corrupts the cross-modal affinity matrix by inverting SAM encoder features under the guidance of CLIP patch-level semantics, and Semantic Interface Poisoning steers the prediction through a max-distance label permutation derived from CLIP text embeddings. Experiments on four collaborative pipelines across eight benchmarks show that CrACK causes catastrophic degradation while every individual model continues to produce its unchanged standalone output, rendering per-model defenses structurally blind. The corruption further cascades into large vision-language model reasoning, driving models such as LLaVA to produce erroneous responses from visually intact inputs. Our results show that the security of a collaborative AI system cannot be reduced to the robustness of its components, and that inter-model feature interfaces must be treated as first-class security boundaries.

Unknown authors · 0 citations