Modern model hubs store hundreds of petabytes of large language models (LLMs), with fine-tuned variants dominating the storage footprint. These variants contain substantial cross-model redundancy that delta compression can exploit by storing only the difference between a target and a reference model. However, compressi...
Ting-Feng Lan, Zirui Wang, Yun-Jia Zheng et al.· Proceedings of the ACM SIGOP...· 0 citations
This work argues for co-design, that facilities should treat agents as first-class principals where agents become facility-aware tenants, and outlines the resulting challenges and opportunities in compute, storage, agent memory, and safety.
Yun-Jia Zheng, Bintang Dwi Marthen, Zachary Pan et al.· arXiv.org· 0 citations
This work studies production traces from two companies and evaluates 14 eviction algorithms across HBM-constrained and large memory-pool settings to show that effective prefix-cache management should retain recency as its foundation while selectively adding quick demotion for one-hit prefixes.
Learning-Augmented Heuristics (LAH), a framework that learns the cache-level parameters of static heuristics, is introduced and demonstrated the effectiveness of LAH through S4-FIFO, a Smart S3-FIFO cache eviction algorithm.
Haocheng Xia, William Nixon, Bintang Dwi Marthen et al.· 1 citation
TrajectoryDB is envisioned, a trajectory-native data management system that co-designs ingestion, storage, and query processing to efficiently manage and analyze agent execution trajectories.
This work introduces HeadWiseKV, a training-free framework that compresses the residual global KV caches of hybrid language models while preserving their native local, recurrent, and linear paths and proposes SeqCalib as the core policy-generation algorithm in HeadWiseKV.
Ren-Jie Xie, Jun-Cheng Yang, Ao-Ting Hu et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.