Deriving Scaling Laws for OpenEuroLLM Models: Learning Rate, Batch Size and Loss
This study establishes a first baseline and scaling procedure for the development of future OpenEuroLLM models, and characterize the dependence of loss on model capacity and dataset size, evaluating recently proposed scaling forms that explicitly model their interaction.
Niccolò Ajroldi, Diana-Alexandra Onutu, Haider Al-Tahan et al.
· 1 citation