GVPO++: Group Variance Policy Optimization for LLM Post-Training and On-Policy Distillation
Group Variance Policy Optimization is introduced, a novel post-training method that integrates the analytical solution of KL-constrained reward maximization into its gradient weighting scheme and establishes a new paradigm for reliable and versatile LLM post-training and on-policy distillation.
Kai-Chen Zhang, Yuzhong Hong, Jun-Wei Bao et al.
· 0 citations