Robust Direct Preference Optimization under Preference Label-Flip Attacks
This work establishes deterministic gradient perturbation bounds uniformly over all attack sets containing at most $K$ flips, explicitly separating defense-induced bias from surviving attack effects, and quantifies how quadratic regularization improves curvature while introducing additional bias.