Effectiveness of Differential Privacy and L2 Regularisation Against Membership Inference Attacks in Deep Learning
TL;DR
It is demonstrated that L2 regularization provides comparable defence against membership inference attacks as DP-SGD, particularly for larger datasets, and that L2 regulation achieves precision, recall, and accuracy values within 2-4% of those achieved by DP-SGD across all experimental conditions.
Abstract
Differential privacy (DP) is a widely used technique for protecting sensitive information in machine learning. A membership inference attack (MIA) attempts to determine whether a specific data record was used to train a model, even when the attacker has no knowledge of the model’s internal structure. This paper empirically investigates and compares the effectiveness of two mitigation techniques—differential privacy via stochastic gradient descent (DPSGD) and L2 regularisation—in defending deep learning models against such attacks. Using a black-box attack framework with shadow models on the CIFAR-10 dataset, we measure attack precision, recall, and accuracy across varying dataset sizes and output formats (probabilistic vs. digital). Our results demonstrate that L2 regularization provides comparable defence against membership inference attacks as DP-SGD, particularly for larger datasets, Specifically, both methods reduce attack accuracy to the range of 0.05-0.06, with L2 regulation achieving precision, recall, and accuracy values within 2-4% of those achieved by DP-SGD across all experimental conditions. While DP-SGD offers formal privacy guarantees with a privacy budget of ε=1.12, it incurs a higher computational cost (approximately 3 times longer training time) and introduces noise into model predictions. In contrast, L2 regularisation—a simple, low-cost method designed to reduce overfitting—emerges as an empirically effective and practical alternative for mitigating membership inference risks without sacrificing utility or performance.