Edge-Cloud Collaborative RAG with Multi-State Semantic Caching

Abstract

Enterprise applications are increasingly adopting collaborative Retrieval Augmented Generation (RAG) frameworks that synergize Small Language Models (SLMs) at the network edge with Large Language Models (LLMs) in the cloud for joint reasoning. However, this paradigm is bottlenecked by the capacity asymmetry between massive cloud-based knowledge repositories and resource-constrained edge servers. Existing edge-cloud collaborative frameworks rely on monolithic storage strategies that fail to exploit the structural asymmetry between the vector index of knowledge chunks and raw documents. This limitation induces a severe conflict between service latency and cache reconfiguration traffic. This paper proposes a Multi-State Semantic Cache (MSSC) framework that dynamically maps knowledge chunks into three mutually exclusive states: zero cache, index cache, and full cache. By leveraging semantic graph diffusion, our framework captures conceptual dependencies to predict query trajectories, enabling predictive state transitions among three states. Furthermore, we formulate the cache scheduling process as a Model Predictive Control (MPC) problem to optimize the trade-off between service latency and reconfiguration traffic. Experimental evaluations on real-world datasets demonstrate that, while strictly preserving generation quality, our framework reduces average service latency by over 40% in dynamic scenarios and suppresses backhaul reconfiguration traffic by up to 38% compared to conventional binary caching baselines.