ReSPO: Reshaped Sequence Policy Optimization for Gradient Starvation in Off-Policy Learning
This work proposes ReSPO (Reshaped Sequence Policy Optimization), a smooth, two-branch sequence-level kernel derived from an $\alpha$-divergence variational objective and an exponential variance-control tilt that effectively learns from long positive reasoning trajectories during early training, even when accumulated p...