R-OPD, a gradient-triggered curriculum that adaptively selects when to revisit initial-student trajectories, improves average accuracy over continued current-policy sampling and produces longer responses, suggesting that well-timed revisits help students use more of their reasoning capacity.
Ling Hu, Tian-Le Xia, Yi-Ding Sun et al.· 0 citations
RouteOPD (Routed On-Policy Distillation), which decomposes local teacher--student disagreement into student-excess sources and teacher-deficit destinations and couples them into explicit transport pairs and demonstrates the effectiveness of explicitly modeling probability transport in on-policy distillation.
Tian-Le Xia, Ling Hu, Yi-Ding Sun et al.· 1 citation
It is shown the contamination is structurally irreversible: removing a source skill after the fact cannot erase the flawed reasoning its descendants have already inherited, so post-hoc rollback recovers only a small fraction of the lost performance, making skill admission a pre-commit necessity rather than a post-hoc f...
Lin-Fang Shang, Ming Xu, Yi-Ding Sun et al.· 5 citations· ⚡1
SPARK is introduced, which uses hidden-state response to diagnose whether a model internally enters an effective reasoning state and to guide lightweight test-time steering, and suggests that susceptibility can serve not only as a diagnostic signal for reasoning failures, but also as a practical guide for targeted test...
BIRD(Bootstrapped Iterative Self-Reasoning Distillation), a two-stage self-reasoning distillation method that improves the rollout distribution before on-policy training and achieves a stronger accuracy-efficiency trade-off than prompting and cold-start on-policy distillation on MATH-500 and AIME benchmarks.
Leichao Dong, Dong-Xu Zhang, Yi-Ding Sun et al.· arXiv.org· 0 citations
This work proposes FADE (FFN Attenuation for DEcoding), a training-free method that attenuates FFN outputs to reduce language-prior dominance and shows that FADE effectively mitigates hallucinations while preserving inference efficiency.
Yichen Guo, Kai Tang, Fenglai Lin et al.· arXiv.org· 2 citations
SeeMe is proposed, a training-free framework that introduces the concept of feature engineering from traditional machine learning into LVLMs and restructures visual tokens through a three-stage token engineering process to suppress hallucination sources while preserving informative visual evidence.
Kai Tang, Jinhao You, Bohua Zhang et al.· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.