This paper conducts a systematic study of unsupervised visual pretraining paradigms that directly leverage visual documents without text extraction, showing that Visual Pretraining is a scalable learner for foundation model intelligence.
Mixed SFT, a single supervised fine-tuning stage that jointly trains on no-CoT and long-CoT data, achieves a clearly higher post-RLVR performance ceiling than next-chunk reasoning RL while requiring over 60 times less training compute.
Yin-Hao Tang, You-Qing Fang, Ya-Nan Sun et al.· 1 citation
Large language models (LLMs) have achieved remarkable performance on high-school and olympiad-style mathematics, yet their capabilities on advanced mathematics remain poorly understood. Existing benchmarks, however, fall short in both scope and evaluation granularity: they provide limited disciplinary coverage and ofte...