The Devil is in the Spectrum Bias: Spectrum-Balanced Feature Matching for Robust Representation Distillation
Spectrum-Balanced Feature Matching, SpecMatch, a simple objective that adaptively emphasizes under-optimized spectral directions while preserving the relative importance of dominant directions is proposed, which improves downstream adaptation across diverse tasks, including image classification, anomaly detection, medi...
Kuniaki Saito, Y. Ushiku
· 0 citations