In speculative decoding, a lightweight draft model proposes tokens that the target model verifies in parallel, speeding up generation without changing the target distribution. D-PACE (Wu et al., 2026) trains such drafters with a loss that weights each position by its contribution to the expected accepted length. Both i...
Tian-Yu Wu, Zhuo-Hang Wang, Jonathan Geuter et al.· 2 citations
CrystalLLM is presented to decouple large-scale execution from the need to access large clusters, enabling engineers to run and observe ranks of interest under faithful large-scale behavior using only a few GPUs.
Shao-Ke Xi, ChonLam Lao, Bo-Yi Jia et al.· Proceedings of the ACM SIGOP...· 0 citations
This work argues for co-design, that facilities should treat agents as first-class principals where agents become facility-aware tenants, and outlines the resulting challenges and opportunities in compute, storage, agent memory, and safety.
Yun-Jia Zheng, Bintang Dwi Marthen, Zachary Pan et al.· arXiv.org· 0 citations
This work studies production traces from two companies and evaluates 14 eviction algorithms across HBM-constrained and large memory-pool settings to show that effective prefix-cache management should retain recency as its foundation while selectively adding quick demotion for one-hit prefixes.