Preprint
Aug 2026
SQuaT: Self-Supervised Knowledge Distillation via Student-Aware Quantized Teacher Features
SQuaT (Student-Aware Quantized Teacher Features), a label-free QAT framework with KD that theoretically eliminates this lower bound on the distillation loss by applying the student's quantization parameters to quantize the teacher's features during distillation is proposed.
H. Lee, Hyeonsik Jo, Jinwook Chung et al.
· 0 citations