Magnitude Profile Pruning: Calibration-Free Structured Attention Head Removal for Transformer Compression
The results show that weight-only statistical scoring can match or outperform data-dependent methods for structured head pruning, providing a practical, zero-cost criterion for Transformer compression.