Sho Takase

Revisiting the Capacity Gap in Chain-of-Thought Distillation from a Practical Perspective

Chain-of-thought (CoT) distillation transfers reasoning behaviors from a strong teacher to a smaller student, but prior work reports a capacity gap: distillation may fail when the teacher-student capability mismatch is large. We revisit the capacity gap from a practical perspective by re-examining commonly used experim...