On-policy distillation (OPD) provides dense teacher supervision on student-generated trajectories, but standard reverse-KL training can assign insufficient probability to other plausible continuations. Teacher entropy alone does not reveal whether uncertainty is concentrated among a few plausible next tokens or dispersed over a long probability tail, nor whether the student already represents those candidates well. Moreover, local teacher probabilities may not predict downstream success. We introduce Sparse Probing and Outcome-calibrated Targets OPD (SPOT), which addresses two coupled decisions, where to probe and what to distill, through an acquisition--exploration--exploitation procedure. During acquisition, a position-level score combines normalized teacher entropy, the probability mass captured by a small top-$k$ candidate set, and student--teacher mismatch to allocate a limited probing budget. During exploration, SPOT evaluates teacher-proposed candidates through verifier-scored student continuations. During exploitation, these outcomes produce a closed-form, KL-regularized target that favors candidates with better downstream outcomes while remaining anchored to the teacher distribution. Extensive experiments across multiple student models and reasoning benchmarks demonstrate the effectiveness of SPOT in improving reasoning performance while balancing solution quality and coverage.
Zikun Qu, Min Zhang, Ming-Ze Kong et al.· 3 citations
WDL-OPD is introduced, a mixture-constrained co-training method with two trainable policies that shows that freezing the auxiliary recovers an anchor-plus-contrast proxy target closely related to OPD$^2$ and W2S-OPD, whereas joint training creates branch-level degrees of freedom that a static delta cannot express.
Zehao Chen, Gong-Xun Li, Tianxiang Ai et al.· 0 citations
We propose LLM-PeerReview, an unsupervised LLM Ensemble method that selects the most ideal response from multiple LLM-generated candidates for each query, harnessing the collective wisdom of multiple models with diverse strengths. LLM-PeerReview is built on a novel, peer-review-inspired framework that offers a transparent and interpretable mechanism, while remaining fully unsupervised for flexible adaptability and generalization. Specifically, it operates in three stages: For scoring, we use the emerging LLM-as-a-Judge technique to evaluate each response by reusing multiple LLMs at hand; For reasoning, we can apply a straightforward averaging strategy or a principled graphical model-based truth inference algorithm to aggregate multiple scores to produce a final score for each response; Finally, the highest-scoring response is selected as the best ensemble output. LLM-PeerReview is conceptually simple and empirically powerful. Our results across four datasets show that the two variants of the proposed approach outperform the advanced model Smoothie-Global by 6.9% and 7.3% points, cross diverse task types including factual recall QA, math reasoning, and instruction following. Notably, we also establish a carefully curated benchmark suite for LLM Ensemble, integrating 12 methods across four classic datasets and three task families, all evaluated under a rigorous and consistent protocol. We hope this repository will help researchers reproduce the LLM Ensemble baselines.
Zhijun Chen, Zeyu Ji, Qianren Mao et al.· arXiv.org· 5 citations
LLMODE is proposed, a token-efficient framework for irregular spatio-temporal forecasting with a frozen LLM backbone that shows competitive overall performance, with clearer advantages under sparse or dynamically complex irregular sampling.
Di Zhang, Jing-Yang Zhang, Zi-Qian Wang et al.· 0 citations
Experimental results show that DEFT-RLVR improves AD reasoning while preserving or even enhancing general visual capabilities, and Autonomous-Driving Multiple-Choice Question (AD-MCQ) provides a flexible, scalable, and extensible foundation for future research on verifiable AD reasoning.
Zi-Xuan Huang, Yang Zhou, Kai-Xuan Wang et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
