Preprint
Aug 2026
Measuring Task-Agnostic Training Data Influence Across Language Model Pretraining
This work proposes a measure of training data influence that does not require selecting a downstream task or validation set as the attribution target, and defines an example's influence by how much its gradient update reduces the squared distance to the final parameters of a given pretraining run.
Yuto Nishida, Hirokazu Kiyomaru, Yusuke Oda et al.
· 0 citations