Jul 2026
Bridging Compute- and Data-Optimal Pretraining
Compute-Data (CD) scaling laws are proposed, a unified framework that bridges compute-optimal scaling, where data scales freely with compute, and data-optimal scaling, where the corpus is fixed while compute can grow without bound.
Tian Qin, K. Hamidieh, David Alvarez-Melis
· arXiv.org · 0 citations