Skip to content

From Solver Trajectories to Teaching Trajectories: Cognition-Aligned Reasoning Distillation

Oct 2026 · 0 citations
Computer Science

TL;DR

CARD is an RL-based framework that transforms solver trajectories into teaching trajectories aligned with the student's cognition that outperforms distillation from raw teacher trajectories in in-domain settings while achieving substantial gains in out-of-distribution performance.

Abstract

Knowledge distillation from large reasoning models (LRMs) often uses raw teacher trajectories as supervision. However, this practice creates a mismatch between the goal of distillation and the nature of teacher trajectories. Advanced LRMs typically improve their reasoning capabilities through reward-driven post-training, such as RLHF. This training optimizes them as strong solvers, rather than as teachers that produce reasoning processes smaller models can easily learn. Since smaller models have different"cognitive capacities"compared to their larger teachers, directly imitating raw teacher trajectories can sometimes be ineffective and often requires a substantial number of high-quality samples. To bridge this mismatch, we propose Cognition-Aligned Reasoning Distillation (CARD), an RL-based framework that transforms solver trajectories into teaching trajectories aligned with the student's cognition. Avoiding expensive resampling, our framework directly transforms existing trajectories. We optimize CARD with two reward signals: (1) a Distribution Alignment Reward that matches the student's predictive distribution, and (2) a Logical Coherence Reward that reduces misleading failed reasoning steps. Comprehensive evaluations across multiple student models and settings demonstrate that CARD outperforms distillation from raw teacher trajectories in in-domain settings while achieving substantial gains in out-of-distribution performance. Code is available at https://github.com/LiteSSLHub/CARD.

View source

Similar papers

#machine learning Preprint Oct 2026

Reward-Aligned Distillation

Reward-Aligned Distillation (RAD), a bilevel framework that weights distillation signals based on their predicted effect on the student's post-update reward, supports reward-guided weighting of teacher supervision across model families and tasks.

Yilan Chen, Zian Su, M. Prajapat et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Transfer-Stratified On-Policy Distillation for RL-Improved Reasoning Teachers

Reinforcement learning can substantially improve a reasoning teacher, but it is unclear which of those improvements survive when the teacher supervises a smaller on-policy student. We study this question in mathematical reasoning by comparing teacher lineages before and after GRPO, multiple student scales, direct GRPO,...

Xiao-Yu Chen, Bo Shao, Tian-Gang Zhu et al. · 0 citations
#natural language process... Open access Sep 2026

TeacherGRPO: Closing the Capacity Gap in Reasoning Distillation via Teacher Alignment

Teacher Alignment is proposed, which directly adapts the teacher toward the student's distribution without discarding data or degrading reasoning quality, and which significantly outperforms baselines across diverse reasoning benchmarks and distillation methods.

Zhen-Yu Lei, Zi-Han Chen, Yao-Chen Zhu et al. · 0 citations

Learning from Think-Mode Advantage via On-Policy Distillation

ThinkOPD outperforms Uniform ThinkOPD in both same-model settings and both cross-model teacher-student pairs, and it exceeds representative rationale and self-distillation baselines in a controlled comparison.

Wan-Qi Ren, Jian-Xiang Wang, Dan-Xuan Liu et al. · 0 citations

Teach Yourself Where to Look: On-Policy Attention Self-Distillation for Reasoning

On-Policy Attention Self-Distillation (OPASD), which complements token-level supervision with solution-conditioned attention distillation, shows that solution-conditioned attention provides a complementary supervision signal that makes on-policy self-distillation more accurate, stable, and compute-efficient.

Safaeid Hossain Arib, Rabeya Akter, I. N. Swapnil et al. · 1 citation

Reward-Aligned Reweighting for On-Policy Distillation

Reward-Aligned Reweighting for On-Policy Distillation (R$^{2}$-OPD), which uses outcome agreement and the magnitude of teacher--student disagreement to continuously reallocate teacher supervision, achieves the highest average accuracy among the compared training methods in both cross-size and same-size distillation.

Hao Xu, Junwei Su, Lan-Song Diao et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.