Wei-Feng Kong

Noise-Aware Shrinkage for Differentially Private Zeroth-Order Fine-Tuning of Large Language Models

SAGE is proposed, a noise-aware shrinkage method that adaptively attenuates privatized estimates according to their estimated signal quality, and shows that shrinkage reduces the quadratic update-risk term faster than the linear descent term, preserving useful descent while limiting the influence of noise-dominated upd...