SIGMA: Self-Improving Alignment Generalization from a Model Spec
SIGMA is proposed, a data generation and training pipeline enabling alignment self-improvement that generalizes to out-of-distribution settings and shows that a Model Spec balancing harmlessness and helpfulness, test-time reasoning for safety deliberation, and high-quality rubrics from SIGMA's task designer agent are c...
Jing-Yu Zhang, Shruti Palaskar, Daniel Khashabi et al.
· 0 citations