D-Quant is proposed, a flexible KV cache quantization framework that introduces a Drift mechanism to convert entropy-coded representations of each token into fixed-size bitstreams, enabling regular memory access and parallel dequantization within attention kernels.
PIVOT, Proxy Indexing Via One full-prefix Traversal, a training-free, drop-in replacement for the DSA indexer that shares one prefix scan across a group of nearby queries, matches the accuracy of the dense DSA indexer while accelerating it by up to 4x and reducing end-to-end latency by up to 1.6x at long context.
Hong Liu, Yuan Cheng, Lin Niu et al.· arXiv.org· 3 citations
Bole is presented, a kernel--runtime co-design that enables efficient tree speculation for hybrid-attention LLMs and couples efficient state management with a batch-wide verification budget calibrated to the complete hybrid forward.
This work provides a scalable and effective framework for extending RLVR beyond the limitations of pattern-based verification to complex, noisy, real-world domains, and generalizes strongly to seven out-of-distribution benchmarks.
Yi Su, Dian Yu, Linfeng Song et al.· Annual Meeting of the Associ...· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.