N. Detlefsen

Six Layers Less: Encoder Pruning for Whisper with Label-Free Recovery

This work presents an approach that ranks encoder layers by the leave-one-layer-out change in Word Error Rate (WER), and presents the pruned model, which is simply a more shallow encoder with fewer layers.