SHIFT-LLM: Distribution Shift Correction in Depth-Pruned LLMs
SHIFT-LLM, a training-free post-pruning correction framework that inserts a Linear Residual Adapter at each pruning site, consistently recovers accuracy lost to depth pruning across most configurations, achieving gains up to +15.7 points on Llama-3.1-8B-Instruct.
Ali Bahri, Hang Li, Hong-Liang Li et al.
· 0 citations