#artificial intelligence
Apr 2026
Self-Distillation as a Performance Recovery Mechanism for LLMs: Counteracting Compression and Catastrophic Forgetting
A performance recovery framework based on Self-Distillation Fine-Tuning (SDFT) that effectively restores model capabilities and offers new insights into the internal mechanisms of self-distillation is introduced.
Chiao-Hsuan Liu, Xin Chen, Xuwen Zhou et al.
· arXiv.org · 0 citations