Reward Budgeting Reduces Premature Convergence in Reinforcement Learning for LLM Reasoning
RL for LLM reasoning often plateaus early as its policy sharpens. Policy entropy is studied as both a diagnostic and a control signal, but increasing it alone does not reliably change the eventual plateau in our experiments. We analyze RL's implicit target policy using a sharpness factor $\kappa$ and effective counterp...