Advantage-Driven Synthetic Curriculum for Reinforcement Learning based Fine-Tuning of Large Language Models

Nathaniel Demchak Pravin Ravishanker Oscar Li

· 0 citations · 4 references

TL;DR

ADSC is introduced, a curriculum learning layer for REINFORCE Leave-One-Out that uses a signal already computed by RLOO to identify prompts near the student’s current learning frontier and uses a multi-armed bandit router to sample from these difficulty buckets.

Semantic Scholar

View source

Similar papers

#natural language process... Jul 2026

LEEPS: Latent-Guided Explore-Exploit Prompt Sampling for Efficient RLVR in Large Language Models

LEEPS is introduced, a Latent-Guided Explore--Exploit Prompt Sampler that adaptively balances the reuse of previously observed informative prompts with continued exploration of uncertain ones, thereby making exploration more targeted without additional rollouts.

Shuang Liang, Hao Zhou, Yifan Gong et al. · arXiv.org · 0 citations

#machine learning Jul 2026

Distilled Reinforcement Learning for LLM Post-training

Extensive experiments show that Distilled RL substantially outperforms standard RL and OPD in terms of both pass@1 and pass@k, and can effectively transfer previously unavailable knowledge from a teacher model to a student model.

Chen Wang, Zhaochun Li, Jionghao Bai et al. · arXiv.org · 2 citations

Jul 2026

Towards Robust Reinforcement Learning for Small-Scale Language Model Agents

A capacity-headroom hypothesis is proposed, which states that PPO performance at the SLM scale depends on both a fluent supervised model and a discriminative reward signal, rather than on the number of model parameters.

Md. Rezwanul Haque, Md. Milon Islam, Fakhri Karray · arXiv.org · 0 citations

Jul 2026

MADA-RL: Multi-Agent Debate-Aware Reinforcement Learning for Parameter-Efficient Reasoning in Compact Models

A controlled study isolates the source of MADA-RL's gains: the counterfactual advantage produces the highest critic improvement rate of any model evaluated, indicating that trained critics learn to correct generator errors rather than to imitate them.

Martino M. L. Pulici, Cuong Xuan Chu, Evgeny Kharlamov et al. · arXiv.org · 0 citations

Preprint Aug 2026

Distill Where You Fail: Recovering Learning Signals of Negative RL-Groups from Adaptive Teacher Guidance

RSTG (Recovering Learning Signals via Adaptive Teacher Guidance), which applies distillation selectively and precisely where it matters most, and substantially outperforms naive GRPO+OPD.

Zhuo-Wen Han, Jinwei Xiao, Zhengxi Lu et al. · 4 citations

Jul 2026

LEACL: LLM-Enhanced Automatic Curriculum Learning for Reinforcement Learning in Long-Horizon Manipulation Tasks

LLM-enhanced automatic curriculum learning (LEACL), a framework that integrates LLMs and ACL to address limitations of ACL, is proposed, which achieves better asymptotic performance in terms of the success rates compared to human-designed dense rewards.

Faraz Heravi, James Ouyang, Zi-Fan Xu et al. · arXiv.org · 0 citations