Distill What the Student Can See: Fisher-Projected On-Policy Distillation for Vision-Language Models
Fisher-Projected On-Policy Distillation (FP-OPD), which distills only locally realizable teacher corrections provide a more effective target for distilling compact vision--language models.