Cross-Resolution Knowledge Distillation for Low-Compute Fine-Grained Bird Classification
TL;DR
The results support cross-resolution KD as a practical way to recover fine-grained information during training while preserving low-resolution inference cost and show that a stronger teacher does not automatically produce a stronger student: transfer quality depends on the balance between teacher supervision strength and student optimization.
Abstract
Fine-grained visual classification relies on subtle local cues and is highly sensitive to input resolution, yet practical deployment often constrains image size and inference cost. Existing low-resolution recognition methods often depend on additional network structures or complex training modules, limiting deployment simplicity. This study evaluates a simple cross-resolution knowledge distillation (KD) strategy under a fixed low-compute constraint. The teacher receives higher-resolution inputs to provide richer class-discriminative visual knowledge, whereas the student is trained at 160×160 and is the only model retained for deployment. Both networks use ResNet-18, allowing resolution-aware supervision to be studied without changing the inference architecture. On the CUB-200-2011 dataset, the 160×160 baseline achieved 66.01±0.53% test accuracy. The 224→160 KD configuration improved accuracy to 70.15±0.63%, while the final 384→160 configuration with T = 4 , α = 0.75 , and a 30-epoch cosine schedule reached 71.60±0.68% accuracy and 71.56±0.62% macro-F1. The results also show that a stronger teacher does not automatically produce a stronger student: transfer quality depends on the balance between teacher supervision strength and student optimization. These findings support cross-resolution KD as a practical way to recover fine-grained information during training while preserving low-resolution inference cost.