Discovering and Preserving Category Correlation Knowledge via Adaptive Reciprocal Knowledge Distillation
Knowledge distillation aims to improve the performance of lightweight student models by transferring knowledge from larger and more powerful teacher models. However, a substantial size gap between teacher and student models often impedes effective knowledge transfer. Most existing approaches adopt a static, one-way tea...