Noa Rubin

RLVR landscapes for iterated multiplications can be benign: Insights from spin-glass theory

This work maps entropy-regularized RLVR over myopic tabular policies onto an energy-based (spin-glass) model over deterministic policies, and finds that a transformer trained from scratch, using only last-token rewards, successfully learns an algorithmic chain of thought for iterated non-Abelian group multiplications.