This paper derives a closed-form optimal linear operator from a small calibration set to reconstruct the activation discrepancy introduced by the pruned layers, and shows that this solution corresponds to the unconstrained optimum of the alignment objective, whereas LinearPatch is restricted to a fixed basis symmetric...
Vincent-Daniel Yun, Junhyuk Jo, Sai Praneeth Karimireddy et al.· arXiv.org· 3 citations
Structured pruning reduces LLM inference costs by removing model components to produce smaller dense models. However, existing methods can cause reasoning performance to collapse on long-form reasoning tasks such as AIME. We propose \textit{LayerTrim}, a training-free structured pruning method that identifies redundant...
Woo-Sang Lim, Haneul Yoo, Vincent-Daniel Yun et al.· 1 citation
KV-cache eviction caps the memory cost of long reasoning traces but is inherently lossy because the model decodes from a partial view of its history. Under aggressive budgets, this not only lowers accuracy but can also cause runaway degeneration, where the model produces incoherent or repetitive tokens until reaching t...
Minsoo Cheong, Woo-Sang Lim, Vincent-Daniel Yun et al.· 0 citations
Results show that HeteroFold enables efficient cross-family KV reuse without receiver prefill, and achieves the best cache-transfer performance on all four long-context benchmarks and most short-context settings.
OptR is proposed, an output-aware rotation method that minimizes post-W_O$ attention-output error and applies an attention-equivalent key reparameterization to reduce large channel-wise offsets without changing the softmax distribution.
Weight-Redundancy Pruning (WRP), a forward-free depth-pruning method that estimates inter-layer redundancy from checkpoint weights to select blocks without calibration data or model forward passes, consistently outperforms existing forward-free magnitude pruning and approaches the performance of activation-based method...
It is shown that inter-layer redundancy can be either localized or globally distributed depending on the LLM architecture, and Representation Locality Score (RLS) is introduced, derived from global inter-layer hidden-state similarity.
Vincent-Daniel Yun, Youngrae Kim, Woosang Lim et al.· arXiv.org· 1 citation· ⚡1
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.