GUPO: Gradient Uncertainty-aware Policy Optimization for Post-Training Large Language Models
Gradient Uncertainty-Aware Policy Optimization is proposed, which models each group gradient as a random variable under a Bayesian formulation and estimates its probability distribution and derives gradient uncertainty using a Dirichlet-based formulation and uses it to calibrate the contribution of each group gradient...
Peizheng Guo, Jian-Qi Zhang, Xing-Yu Zhang et al.
· 0 citations