Two-Stage NAS for On-Device Privacy Preservation Across Heterogeneous Computational Resources

Abstract

Edge devices handle sensitive user data, raising significant security and privacy concerns when shared with data collectors for downstream learning tasks. One potential solution leverages differentially private generative models, keeping original data on user devices and creating obfuscated variants for transmission. However, this approach presents two challenges: 1) Obfuscated data that closely resembles the original compromises privacy, whereas excessive dissimilarity diminishes its utility. 2) Diverse computing capabilities of edge devices hinder deploying such models across hardware platforms. To address these issues, we first introduce Differential Privacy with Adaptive Clipping and Noise Scaling (DP-ACNS), which dynamically adjusts privacy parameters to better balance privacy and utility than conventional DP training. Next, to overcome deployment challenges, we propose a two-stage Neural Architecture Search (NAS) approach. In Stage 1, we utilize DP-ACNS to centrally train an over-parameterized network on proxy data. Following this, we iteratively apply symmetric pruning with subsequent knowledge distillation to generate pretrained architectures. In Stage 2, we conduct a feedback-driven evolutionary search to identify optimal architectures that meet edge computational constraints, and adapt them via minimal fine-tuning on target devices. Experimental results demonstrate that our approach effectively balances privacy and utility while maintaining performance across diverse computational environments in edge computing.