Jia-Yu Miao

An empirical study of teacher supervision failure in adversarial distillation for lightweight visual recognition

Adversarial distillation transfers the robustness of a strong teacher to a lightweight student, and most methods implicitly assume the teacher is reliable at every adversarial point. We test this assumption empirically. We define teacher supervision failure as the event that the teacher’s top-1 prediction on an adversa...