SAC-RAG: Semantic Adaptive Context Compression for Retrieval-Augmented Generation

Abstract

Retrieval-Augmented Generation (RAG) effectively mitigates the knowledge cutoff and hallucination issues of large language models by incorporating external knowledge bases. However, standard RAG applies a uniform retrieval strategy to all question types and feeds redundant retrieval results directly into the generator, leading to high token consumption and potentially degraded answer quality. This paper proposes SAC-RAG (Semantic Adaptive Compress RAG), a unified framework that integrates question-type-adaptive retrieval with context compression. SAC-RAG first classifies queries into three types—factual, definitional, and reasoning—via an LLM classifier, applies a differentiated retrieval strategy for each type, then uses a unified LLM context compression module to extract evidence snippets, and finally generates answers from the compressed and refined context. To independently quantify the contribution of each component, we design four progressive ablation experiments on three multi-hop QA datasets—HotpotQA, 2WikiMultiHopQA, and MuSiQue—and evaluate using four metrics: Exact Match, F1, Recall@10, and token consumption. Experimental results show that SAC-RAG reduces token consumption by 38%–58% at the cost of only a 1–2 percentage point EM drop, with EM actually improving after compression for reasoning-type questions, achieving the optimal quality–efficiency trade-off in terms of token consumption, with a moderate increase in per-query latency due to additional LLM calls. SAC-RAG offers a modular and interpretable optimization solution for the efficient deployment of RAG in real-world settings.