Jul 2026· IEEE International Conference on Cloud Computing· pp. 246-256· 0 citations· 41 references
Abstract
Modern serverless platforms create function instances based on user-specified memory limits, which often result in memory over-provisioning even when the average runtime footprint is small. This over-provisioning reduces warm container density under a fixed memory budget, leading to early pool saturation, frequent evictions, and increased cold-start latency. To address this fundamental limitation, we propose Ditto, a system that integrates online memory rightsizing with value-aware container management. Ditto dynamically adjusts container memory to match actual usage, reclaiming unused memory while safely avoiding out-of-memory (OOM) conditions. In addition, it prioritizes containers based on their benefit per unit memory using a score-based eviction and keep-alive policy. We implement Ditto in Apache OpenWhisk and evaluate it across diverse workloads. Ditto reduces cold-start rate by up to 25.7% and latency by up to 33.3% compared to vanilla OpenWhisk, and by up to 14.8% and 19.4%, respectively, compared to RainbowCake.
Log-structured merge-tree (LSM-tree) key-value stores rely on caching to mitigate multi-component lookups and long tails, yet block and KP caches are prone to compactioninduced expiry, and all three cache types suffer from scan pollution under LRU eviction. We present AutoThermKV, a two-level in-memory architecture tha...
Yunfan Chi, E. Sha, Longshan Xu et al.· IEEE International Conferenc...· 0 citations
The root causes are examined, a decomposition model is built, and three mitigation strategies are evaluated: memory snapshotting, fork-based execution (REAP), and Firecracker snapshot restore are evaluated.
Satish Chavali· Journal of Scientific Engine...· 0 citations
FaaSlim is proposed, a snapshot-based serverless computing system that integrates both in-memory caching and snapshot-based in-storage caching to reduce cold start latency, and devise a partial caching-aware eviction policy, GDSF-CE, which selects VMs and page subsets to evict based on cache efficiency.
Sanghyeon Eom, Chanyoung Park, Gihong Lee et al.· International Conference on...· 0 citations
This work presents Oneiros, a dynamic remapping engine for multi-tenant LLM serving that dynamically repurposes GPU memory allocated for model parameters as KV cache capacity, enabling nonblocking, unidirectional parameter transfer.
Results show that object-value signals rank what to retain, while persistent responsibility determines which group bears reclamation pressure, which shows that object-value signals rank what to retain, while persistent responsibility determines which group bears reclamation pressure.
Large language model serving faces a critical memory bottleneck: the KV cache grows with sequence length and batch size. PagedAttention uses fixed-size memory blocks to reduce allocator-level fragmentation, but recent KV eviction algorithms operate at a token granularity finer than block-level management. This mismatch...
Yuanhang Gao, Xiangrui Yang, Yuan-Feng Chen et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
