Adversarial Training and Differential Privacy-Style Noise Injection for Privacy-Preserving Vertical Federated Learning

Abstract

The adoption of federated learning (FL) has been on the rise in recent years due to the decentralized approach to data handling. Vertical federated learning is a type of FL that allows different parties to train shared models on complementary feature spaces without the direct exchange of data. However, the gradients these parties exchange can inadvertently carry sensitive information. Adversaries exploit this leakage to mount label inference attacks (LIAs) and adversarial attacks. To curb this, defense mechanisms have been deployed, but most of them either trade robustness for privacy and model utility or vice versa. This study addresses this gap by introducing an improved defense mechanism that combines adversarial training (to harden the model against adversarial perturbations) and differential-privacy-style noise injection (aimed at restoring the label privacy weakened by adversarial training) to collectively enhance the robustness of the existing KDk defense mechanism with marginal model utility trade-off. Instead of relying on heavy encryption or post-processing techniques, it builds privacy directly into the learning dynamics of the model. It was evaluated using five publicly available datasets spanning three data modalities with the proposed mechanism achieving competitive near-baseline accuracy while significantly reducing label-inference success. Under FGSM-based adversarial evaluation, the robustness gap of this mechanism was found to be approximately 1% compared to the 36% robustness gap of the existing KDk mechanism. The Privacy Leakage Index (PLI) reached 81.32%, 96.08%, 82.41%, 86.68% and 73.88% for CIFAR-10, CIFAR-100, CINIC-10, Yahoo! Answers and Criteo datasets, respectively. The results suggest that robustness and privacy security objectives can coexist to secure VFL with minimal effect on model accuracy.