Multi-layer Adversarial Robustness Analysis of Neural Networks: Visual and Metric-based Approaches

Abstract

Adversarial examples are purposely crafted to fool Artificial Neural Networks (ANNs) into producing erroneous outputs. Understanding what makes these models vulnerable to such attacks is an open research topic and constitutes a first step toward devising defenses. Focusing on ANNs designed for image classification, we propose a method to analyze these models from an adversarial robustness perspective. The approach is based on the t-distributed Stochastic Neighbor Embedding (t-SNE) technique and allows the visual inspection of the representations learned by an ANN throughout its different layers. Additionally, we propose the Embedding Overlap Robustness RatiO (EORRO) metric to identify potential weak spots in the network by quantifying differences between clean and perturbed images on the t-SNE maps. We validated our approach by studying two ANNs, one designed by humans and another by NeuroEvolution, on CIFAR-10 and CIFAR-100. Our results indicate that differences between clean and perturbed images start to emerge early on in the models, namely in the feature extraction layers of the networks. Guided by the proposed metric, we implemented an ablation and fine-tuning strategy that successfully boosted the robustness of the ANNs against a variant of the Auto-PGD attack under different threat models. More specifically, the post-attack accuracy of the neuroevolved ANN on CIFAR-10 changed from 0% to at least 63.5% in \(L_{2}\) and 27.2% in \(L_{\infty}\) .