Compute-Data (CD) scaling laws are proposed, a unified framework that bridges compute-optimal scaling, where data scales freely with compute, and data-optimal scaling, where the corpus is fixed while compute can grow without bound.
Tian Qin, K. Hamidieh, David Alvarez-Melis· arXiv.org· 0 citations
MatrAIx is introduced, a population-scale simulated-user evaluation infrastructure for testing AI systems and digital products with heterogeneous users and provides an end-to-end infrastructure for evaluating AI systems and digital products with diverse simulated human users.
Xiaomin Li, Yuexing Hao, Jian Hou et al.· 1 citation
This work formalize and quantify data synergy in language model pretraining by leveraging observational variation across open-weight LLMs with diverse pretraining mixtures and estimating both direct domain-to-benchmark synergy and second-order domain-domain synergy (capabilities that require co-occurrence of multiple domains).
K. Hamidieh, Lester Mackey, David Alvarez-Melis· arXiv.org· 4 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.