Knowing but Not Saying: Preventing Factual Access Failures in LLM SFT via Recall-Anchored Distillation
Recall-Anchored Distillation (RAD), a base-anchored self-distillation objective that preserves out-of-distribution generation behavior by aligning the adapted model with the original base model's soft continuation distribution on unlabeled OOD text, is introduced.