Reinforcement learning objectives that maximize expected return can concentrate probability mass on a small subset of high-reward outcomes, limiting diversity when multiple distinct solutions are desirable. We introduce Density-Aware Reward Scaling (DARS), a simple reward-processing mechanism that promotes reward-propo...
Ria Shekhawat, A. Sinha, Ying-Tao Zhu et al.· 0 citations
Expected-reward maximization in reinforcement learning concentrates probability mass on a few outcomes, even in tasks that require many distinct, high-quality solutions, a failure known as mode collapse. Inverse Probability Scaling (IPS) counters this in group-based policy gradients such as GRPO by dividing each reward...
Data-DPO, a target model-oriented SFT data selection method that consistently outperforms existing data selection baselines under multiple data budgets and stably surpasses full data training performance is proposed.
Peng Sun, Yi Yang, Antong Zhang et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.