Shu-Lei Wang

Learning Representations through Token Prediction: Geometry, Approximation, and Downstream Guarantees

A statistical framework connecting token prediction with representation geometry, encoder approximation, and downstream performance is developed, introducing a self-consistency principle showing that repeated applications of a shared representation block can progressively refine the contextual representation without introducing additional block parameters.