We have 5 of 10 papers
We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.
Not the right person? Other researchers publish under this name.
On-policy distillation (OPD) trains language models using dense token-level teacher supervision on student-generated trajectories. However, its reliance on frequently refreshed student rollouts often incurs substantial generation cost. We introduce REVO, an off-policy distillation framework that improves rollout effici...
An RL perspective on OPD is provided that offers both a principled interpretation and a practical route toward more effective and rollout-efficient language model distillation.
CIS replaces the unbounded second moment that governs the error of exact importance sampling with a term bounded by a constant, at the cost of a bias controlled by the truncated excess, in evaluation across three mixture-of-experts models and five mathematical reasoning benchmarks.
We study length inflation in on-policy distillation (OPD), where student responses can become excessively long and even exhaust the generation budget. We identify \emph{termination-token mismatch} between base students and post-trained teachers as an important source of this behavior. Across Qwen3, Llama, and Gemma, th...
TIGER is presented, an inference-time framework that redesigns feedback for localized repair that reduces unsupported content while preserving task quality and a CrisisFACTS case study suggests that the same repair mechanism can improve grounding in multi-source settings.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
