COEC: Calibrated Orthogonal-Equivalence Compensation for Structured Pruning of Large Language Models
COEC (Calibrated Orthogonal-Equivalence Compensation), a training-free compensation framework that applies alternating left and right orthogonal rotations to the retained weight, improves perplexity on every model and zero-shot accuracy in most settings over existing compensation methods.
Peiqi Yu, Nam Ling, Wei Wang et al.
· 0 citations