Predicting how model propensities generalize out of distribution
How do language models generalize from their training data? This is a central challenge for alignment: models reliably learn their training distribution given enough samples, but we want models that behave well when deployed in new contexts. We study out-of-distribution (OOD) generalization of propensities. Our central...
J. Hunter, Li-Sheng Wen, Owen Terry et al.
· 0 citations