Improving cross-architecture black-box adversarial transferability via hybrid structure-aware feature perturbations.

Abstract

Transfer-based black-box adversarial attacks provide a practical means to evaluate the robustness of deep neural networks under restricted access to target models. However, existing approaches suffer from severe performance degradation in cross-architecture scenarios, particularly when adversarial examples crafted on Vision Transformers (ViTs) are transferred to convolutional neural networks (CNNs). We argue that this limitation cannot be fully attributed to optimization strategies alone, but may also be related to the mismatch between perturbation structure and model-specific inductive biases. To address this issue, we propose a structure-aware adversarial perturbation refinement framework that explicitly enforces spatial coherence during forward propagation. The proposed method consists of three components. First, spatial autocorrelation analysis is used to guide perturbation allocation toward structurally discriminative regions. Second, spatially connected perturbation patterns are introduced to help preserve perturbation effects under convolutional smoothing and pooling operations. Third, the perturbation strength is adaptively adjusted across network depth to balance structural disruption and semantic preservation. Extensive experiments on the ImageNet benchmark show that the proposed approach achieves competitive and often stronger performance than the compared transfer-based attacks across diverse ViT and CNN architectures, with particularly notable gains in the challenging ViT-to-CNN transfer setting. These results provide empirical support for the utility of structure-aware perturbation refinement in improving black-box adversarial transferability across heterogeneous visual architectures.