Memory disaggregation provides key-value stores larger memory capacity at low cost. Emerging compute express link (CXL) enables efficient memory disaggregation. It, however, dramatically slows down the system performance as disaggregated memory accesses are considerably slower than local memory accesses. This paper presents Nemo, a near-memory accelerator architecture for CXL-based memory-disaggregated key-value stores. To address the CPU-accelerator latency, Nemo provides a set of new techniques that enable fast task dispatch, cache-conscious task offloading without CPU modifications, and versatile support of custom data types. We evaluate Nemo on the core in-memory index of RocksDB and six widely used in-memory indexing microbenchmarks under YCSB query workloads. Nemo reduces up to 91.8% of slow disaggregated memory accesses, achieves a 2.05 × average speedup over the CPU-only configuration, and accelerates key-value stores by up to 4.5 ×.
Chencheng Ye, Yuanchao Xu, Xipeng Shen et al.· ACM Transactions on Architec...· 0 citations
Serverless computing has emerged as a compelling cloud paradigm due to its simplified development model, automatic scalability, and fine-grained billing. While its stateless execution model enables high elasticity and resource efficiency, it poses noteworthy challenges for building complex stateful applications. To bridge this gap, modern serverless platforms rely on external cloud storage systems to manage persistent state, giving rise to stateful serverless computing. However, this decoupled architecture introduces two fundamental limitations: performance bottlenecks caused by frequent remote state accesses, and state reliability issues stemming from function crash retries. We propose Arcus, a high-performance function state I/O system co-designed for both idempotent execution and low-latency access. Arcus integrates two key techniques: CFLog, a coordination-free logging mechanism that eliminates internal ordering overhead while ensuring idempotent function execution, and FCCache, a cache system optimized for the dispersed invocation pattern of serverless workloads, which reduces read/write latency by improving local hit rates for reads and shortening the write path. Compared to the state-of-the-art system Halfmoon, Arcus reduces read/write latency by 80.4% and 87.9%, respectively. In evaluations using three real-world applications, Arcus achieves an average 60.3% reduction in end-to-end request latency, demonstrating its effectiveness for fast and reliable function state I/O.
Yijie Liu, Zhuo Huang, Hanxiang Huang et al.· IEEE Transactions on Paralle...· 0 citations
Retrieval-Augmented Generation over Knowledge Graphs (GraphRAG) enhances Large Language Models (LLMs) with structured, multi-hop evidence. However, existing GraphRAG systems predominantly linearize retrieved subgraphs into long textual prompts, forcing LLMs to recompute identical schema-level reasoning across queries repeatedly. This text-centric design incurs substantial prefilling latency, memory overhead, and severely limited cache reuse under entity-level variations. We observe that although retrieved entities differ across queries, their underlying logical schemas (meta-structures) recur with high frequency, indicating that most computational cost is spent on repeatedly encoding invariant structural logic. In this paper, we propose MetaKV, the first structure-aware KV caching mechanism that explicitly decouples static structural logic from dynamic entity semantics in GraphRAG inference. In a preparation phase, MetaKV mines frequent meta-structures and pre-computes their Key-Value (KV) caches as reusable Skeleton KVs. During inference, query-specific entity representations are injected into reserved structural slots to assemble the context without recomputing graph topology. To further enforce faithfulness to graph reasoning, MetaKV introduces a Topological Mask that constrains attention to valid graph edges. Extensive experiments conducted on HotpotQA and MetaQA datasets demonstrate that MetaKV achieves up to 6.4× prefilling speedup and a 73% effective cache-hit rate while maintaining competitive reasoning accuracy, enabling high-throughput, low-latency GraphRAG without sacrificing adherence to graph topology.
Ruikun Luo, C. Gu, Jing Yang et al.· Proceedings of the 32nd ACM...· 0 citations
MetaKV is proposed, the first structure-aware KV caching mechanism that explicitly decouples static structural logic from dynamic entity semantics in GraphRAG inference, enabling high-throughput, low-latency GraphRAG without sacrificing adherence to graph topology.
Ruikun Luo, C. Gu, Jing Yang et al.· Proceedings of the 32nd ACM...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
