Vertumnus is presented, an adaptive CP serving system designed for heterogeneous and evolving workloads that reduces mean TTFT and improves token-weighted SLO attainment and introduces a global prefix-cache management policy that coordinates cache placement and replication among workers with the same or different CP de...
Jiarui Guo, Rong-Le Wang, Pei-Jun Huang et al.· 0 citations
SCOPE is a training-free sparse attention framework that combines 3D-RoPE-aligned key subspace clustering with online per-head Top-$k$ estimation for efficient video-DiT inference and consistently outperforms existing training-free baselines in both fidelity and latency.
Qi Zhao, Qi-Rui Li, Hanlin Tang et al.· 0 citations
Shiva-DiT, based on Residual-Based Differentiable Top-k Selection, provides four fidelity-latency operating points and reaches a 1.54x wall-clock speedup with competitive fidelity.
C2KV is proposed, a unified framework for non-prefix KV reuse that jointly optimizes KV cache compression and concatenation that significantly reduces KV cache storage and transfer costs.
Chuheng Du, Jun-Yi Chen, Hanlin Tang et al.· Proceedings of the 32nd ACM...· 2 citations
A causal interpretability framework is introduced that reveals that the tokens encoding semantics at the input need not be those that maintain them during generation, but also provides a causal view of internal mechanisms in diffusion transformers.
Generative retrieval has become an increasingly adopted paradigm for industrial search, recommendation, and advertising systems, delivering significant online gains. Most existing work combines user behavior sequences with large language models (LLMs) to model user preferences. In practice, feature engineering remains...
Dan Ou, Gui Ling, Haokai Wan et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
