Enterprise document security-level classification must balance rare high-risk recall, precision, and inference cost. Static ML pipelines lose recall as sensitivity patterns drift, whereas retrieval-augmented classification (RAC) grounds predictions in auditable precedents but still struggles on boundary cases, and veri...
Yeseul Chang, Woo-Sang Lim, Rahul Kailasa et al.· Proceedings of the 35th ACM...· 0 citations
EntropyCache is proposed, a training-free KV caching method that uses the maximum entropy of newly decoded token distributions as a constant-cost signal for deciding when to recompute, independent of context length and model scale.
As personal AI agents act on users' email, files, and payments, SPILLWAY gives them an auditable refusal floor on where their actions go that keeps them useful and holds across backbones.
Woo-Sang Lim, Yeseul Chang, Jaewoo Lee Saint Machines et al.· 0 citations
Structured pruning reduces LLM inference costs by removing model components to produce smaller dense models. However, existing methods can cause reasoning performance to collapse on long-form reasoning tasks such as AIME. We propose \textit{LayerTrim}, a training-free structured pruning method that identifies redundant...
Woo-Sang Lim, Haneul Yoo, Vincent-Daniel Yun et al.· 1 citation
KV-cache eviction caps the memory cost of long reasoning traces but is inherently lossy because the model decodes from a partial view of its history. Under aggressive budgets, this not only lowers accuracy but can also cause runaway degeneration, where the model produces incoherent or repetitive tokens until reaching t...
Minsoo Cheong, Woo-Sang Lim, Vincent-Daniel Yun et al.· 0 citations
Results show that HeteroFold enables efficient cross-family KV reuse without receiver prefill, and achieves the best cache-transfer performance on all four long-context benchmarks and most short-context settings.
OptR is proposed, an output-aware rotation method that minimizes post-W_O$ attention-output error and applies an attention-equivalent key reparameterization to reduce large channel-wise offsets without changing the softmax distribution.
Weight-Redundancy Pruning (WRP), a forward-free depth-pruning method that estimates inter-layer redundancy from checkpoint weights to select blocks without calibration data or model forward passes, consistently outperforms existing forward-free magnitude pruning and approaches the performance of activation-based method...
It is shown that inter-layer redundancy can be either localized or globally distributed depending on the LLM architecture, and Representation Locality Score (RLS) is introduced, derived from global inter-layer hidden-state similarity.
Vincent-Daniel Yun, Youngrae Kim, Woosang Lim et al.· arXiv.org· 1 citation· ⚡1
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.