Entropy Regularization: A Free Correction to Cross-Entropy for Verified Demonstrations
It is shown that minimizing cross-entropy can be misaligned with minimizing verifier risk; two policies can assign identical likelihood to the observed demonstrations while placing different probability mass on incorrect outputs, and entropy-regularized training consistently improves verifier accuracy over standard cro...
Mihir Dhanakshirur, Adam Ousherovitch, A. Tewari
· 0 citations