Puro-2B: Poor Lab's Qwen2-1.5B Trained on RTX 5090 within $5090
This report presents an open pretraining recipe that trains a collection of Puro-2B models from scratch on up to 1.4 trillion tokens with FP8 precision on consumer-grade RTX 5090 GPUs, and derives a Puro Cost Scaling Law that relates training cost to average model performance.
Kairong Luo, Jia-Rui Cui, Yao-Rui Yin et al.
· 0 citations