MC-SNN: Multicenter Stochastic Neural Network for Adversarially Robust Learning.
TL;DR
A multicenter learning method that leverages the advantage of stochastic neural networks (SNNs) for feature uncertainty learning and induces multiple centers for each class of samples in latent space to fit data more delicately, named the multicenter SNN (MC-SNN).
Abstract
Enhancing the adversarial robustness of deep neural networks (DNNs) has become a prominent topic in the field of reliable artificial intelligence. Existing methods, no matter with standard training (ST) or adversarial training (AT), usually adopt a regular learning mode that induces a single center for each class of samples in the logit layer. However, due to the complex nature of data, a class of samples may have multiple dense regions, thereby following a mixed Gaussian distribution in latent space. The single-center learning mode makes it difficult for the network to fit data at a fine-grained level, thereby increasing the risks of some samples being located near the classification boundary, which degrades the model's adversarial robustness. In this article, we propose a multicenter learning method for robustness enhancement. It leverages the advantage of stochastic neural networks (SNNs) for feature uncertainty learning and induces multiple centers for each class of samples in latent space to fit data more delicately, named the multicenter SNN (MC-SNN). In addition, four AT-related strategies are introduced to propose MC-SNN-AT, in order to defend against a wider range of attacks. In a series of benchmark tests, both MC-SNN and MC-SNN-AT achieved state-of-the-art robustness. Furthermore, the training cost of MC-SNN is only about one-tenth of vanilla AT.