Structural priors for data-efficient language learning
Non-language data can serve as a partial substitute for language data for the training objective of next-token prediction but does not reliably support broader linguistic generalization, and transfer from non-language data is less efficient than additional language data.
Yana Veitsman, Jonas Mayer Martins, Jonathan Lautenschlager et al.
· 1 citation