Lightweight vision transformer for adversarially robust object detection in autonomous vehicles

Abstract

The increasing deployment of autonomous vehicles (AVs) in intelligent transportation systems necessitates perception models that are not only accurate but also robust and computationally efficient. Although Vision Transformers (ViTs) have shown strong performance in visual recognition tasks, their high computational complexity and vulnerability to adversarial perturbations limit their applicability in real-time and safety-critical autonomous driving scenarios. This paper presents a Lightweight Vision Transformer (L-ViT) framework for robust object detection in autonomous vehicles. The proposed approach integrates a lightweight transformer backbone with a deformable attention-based detection head and introduces a detection-aware Adversarial Consistency Regularization (ACR) strategy. Unlike conventional feature-level defenses, the proposed ACR enforces prediction consistency at the object query, classification, and bounding box regression levels, explicitly embedding robustness into the object detection pipeline. The proposed framework is evaluated on three widely used autonomous driving benchmarks, including KITTI, BDD100K, and Cityscapes. Experimental results demonstrate that the proposed L-ViT achieves competitive detection accuracy while maintaining real-time inference performance on edge devices. Under adversarial conditions, the proposed method significantly reduces performance degradation, exhibiting an accuracy drop of approximately 13\% under strong gradient-based attacks, compared to drops exceeding 25% for conventional detectors. These results indicate that the proposed L-ViT framework effectively balances detection accuracy, adversarial robustness, and computational efficiency. By integrating lightweight architectural design with robustness-aware training, this work provides a practical and deployable solution for secure perception in real-world autonomous driving systems.