Open access
Aug 2026
Resource-Efficient Pruning for Transformer via Low-Rank Importance Estimation
REP-LIE leverages the gradients of LoRA low-rank matrices to estimate the importance of weights without requiring full gradient computation, and a stability score is introduced, serving as the basis for iterative pruning of unimportant model parameters.
Peng Liu, Hui-Bing Zeng, Yi-Qun Zhang et al.
· IEEE Transactions on Emergin... · 0 citations