Pick Up Where You Left Off: An Efficient Solution for Continuous Vector Similarity Search

Abstract

Efficient vector similarity search is critical for Retrieval-Augmented Generation (RAG) systems and other real-time AI applications. However, most existing methods are optimized for isolated queries and fail to leverage the continuity and correlation inherent in real-world query streams, such as those in multi-turn dialogues and multi-hop question answering. We formalize this problem as Continuous Vector Similarity Search (CVSS). While recent efforts attempt to reuse prior results, they either sacrifice accuracy through semantic caching or yield only marginal efficiency gains. To address this, we propose Reuse, an end-to-end framework that decomposes CVSS into two synergistic components: (1) Reuse Trigger that decides when to reuse prior search results, and (2) Reuse Searcher that addresses how to reuse them effectively. Together, these components significantly reduce redundant computation while maintaining near-identical recall. Extensive experiments on four real-world datasets show that Reuse achieves 1.6--3.0× higher throughput (QPS) than state-of-the-art methods at the same recall.