Preprint
Aug 2026
Privileged, but Biased: How PI-Conditioned Teachers Break Self-Distillation
This work reproduces SDPO's reported gains in its easy setting, then applies the identical setup to difficult tasks and finds that it does not teach anything, and explains this failure through a single causal chain from the loss to the model it produces.
Sarthak Harne, Chinmay Karkar, Yash Pandya et al.
· 6 citations