Self-Play Pretraining with Zero Data
Self-Play Pretraining with Zero Data is introduced, an initial proof-of-concept towards realizing a more general approach to pretraining, and whether zero-shot performance on natural data improves predictably with self-play compute is tested.
Aditya Cowsik, Kfir Dolev, Michael Y. Li et al.
· 1 citation