Hemant Pugaliya

Multi-Scale Model Compression via Nested Matrix Learning

Nested Matrix Learning is proposed, a method that trains a single, flexible model capable of generating multiple high-performing “student” models of varying sizes by simultaneously optimizing a pre-trained teacher model and its nested sub-models in a single training process.