Evaluating need for adversarial training data given algorithmic defense methods against adversarial attacks

Abstract

An adversarial attack is a modification to the pixels of an image for the purpose of making a machine learning system misclassify the image. The foremost defense against adversarial attacks is adversarial training: a process in which the machine learning system trains on the already attacked images. But, this is not the only kind of defense. There are also algorithmic defense methods, which work to modify the learning process to be resilient to adversarial attacks without involving attacked examples. In this study, we considered three algorithmic defense settings: no algorithmic defense, defensive distillation, and gradient masking. Then we evaluated the role of adversarial training as part of defending machine learning models from adversarial attacks. Specifically, we set up a baseline image classifier for images of digits (MNIST dataset) and attacked the images using the fast gradient sign method. We hypothesized that introducing adversarial training for this classifier would significantly improve downstream classification accuracy in all three algorithmic defense settings. We found that, for all algorithmic defense settings applied to neural networks with between one and six convolutional layers, adding adversarial training consistently resulted in a statistically significant increase in accuracy. While these findings are limited by the specific data, parameters, and algorithms explored, our results suggest that implementing adversarial training within all lines of defense against adversarial attacks would be beneficial. We believe that this insight increases awareness of cybersecurity threats such as adversarial attacks and lowers the barrier of entry to protect against them.