Mohamed Echchidmi

Teacher Promotion and Progressive Shrinking: a Framework for Iterative Distillation

Deep neural networks often achieve high accuracy, but their compute, memory, and power demands can hinder deployment on resource-constrained edge devices. This paper proposes an iterative, constraint-aware knowledge distillation framework that compresses a large model into a small one under an explicit model-size budge...