Preprint
Aug 2026
Information Abundance Paradox: Long-Context Training Undermines Parametric Knowledge
The Information Abundance Paradox is supported and it is suggested that scaling toward near-infinite context is not simply a matter of supplying more data, even when high-quality long-context data is abundant.
Arda Uzunouglu, Benjamin Van Durme, Daniel Khashabi
· 2 citations