Toward Improving Stochastic Neural Network Robustness via Arbitrary Distribution Injection.
TL;DR
A novel SNN named arbitrary distribution injection is proposed, which enables distribution modeling from nonpredefined, data-dependent distributions and introduces a conditional stochastic feature mapping mechanism to model feature distributions, together with a theoretically grounded variance-regularization loss.
Abstract
Adversarial attacks pose significant challenges to the security and robustness of deep-learning models. Stochastic neural networks (SNNs) have shown promising effectiveness in improving robustness by injecting stochastic noise into model activations, features, or weights. However, most existing SNN-based defenses rely on predefined distributional forms, such as Gaussian or Uniform. In real-world scenarios, data distributions are often non-Gaussian, skewed, or multimodal, which cannot be adequately captured by such fixed assumptions, thereby limiting the robustness of existing methods. To address this limitation, we propose a novel SNN named arbitrary distribution injection (ADI), which enables distribution modeling from nonpredefined, data-dependent distributions. In particular, we introduce a conditional stochastic feature mapping mechanism to model feature distributions, together with a theoretically grounded variance-regularization loss. Extensive experiments across diverse attack methods, datasets, modalities, and network architectures show that ADI achieves robustness improvements and promising generalization across the evaluated settings. Furthermore, detailed parameter analyses and feature-distribution visualizations provide deeper insights into the underlying mechanisms of ADI.