Martin Wattenberg

Beyond the Rosetta Stone: Unification Forces in Generalization Dynamics

This work introduces a controlled setting to study the causes and training dynamics of cross-lingual knowledge transfer by training small Transformer models from scratch on synthetic multilingual datasets and suggests methods to encourage representational unification as part of training that would improve LLMs'cross-lingual transfer.