REER-PT: Reverse-Engineered Reasoning for Perplexity-Guided Pre-training Data Augmentation
Together, the perplexity analysis indicates improved continuation predictability, while the controlled pre-training experiments suggest that this augmentation can improve model performance without changing the standard pre-training objective.
Haoran Que, Jia-Jun Shi, Ting Huang et al.
· 0 citations