Reinforcement learning (RL) has become increasingly important for improving the reasoning, instruction-following, and task-solving capabilities of large language models (LLMs). However, scaling RL training exposes failures in distributed execution, feedback computation, and learning dynamics that can waste computationa...
Ling-Zhe Zhang, Tong Jia, Chi-Ming Duan et al.· 0 citations
Reinforcement fine-tuning (RFT) is increasingly used in applications where large language models (LLMs) interact with humans and other agents. Here we use social deduction games to study how RFT changes LLMs'social behaviour. We let fine-tuned and base LLM agents play hidden-role games that require hidden-state inferen...
Ling-Zhe Zhang, Yun-Peng Zhai, Tong Jia et al.· 0 citations
This work proposes RLCSD (Reinforcement Learning with Contrastive on-policy Self-Distillation), which mitigates this drift by contrasting the teacher-student gap under a correct hint against that under a wrong hint, suppressing style shifts induced by hints regardless of correctness and yielding a signal more concentra...