Progress-Aware Distillation for Mitigating Stagnation in Small Language Model Agents
Progress-Aware Distillation (ProD) is proposed, which explicitly detects and penalizes stagnation in student-generated trajectories and encodes the stagnation gap between student and teacher into an adaptive margin for dynamically penalizing possible stagnation.
Yuan-Pu Cao, Saket Sathe, Hanyu Wang et al.
· 0 citations