OneDB: A Distributed Multi-Metric Data Similarity Search System
TL;DR
OneDB is presented, a distributed framework for multi-metric similarity search that integrates data management with learning-based techniques and captures user intent effectively, achieving 12.63%--30.75% higher accuracy and 2.75× faster retrieval speeds compared to state-of-the-art vector search systems.
Abstract
The rapid accumulation of multi-modal data (e.g., text, images, and geo-locations) presents significant opportunities for data mining applications in healthcare and e-commerce. However, effectively retrieving such data remains challenging due to the difficulty in capturing diverse and dynamic user retrieval intents. Existing solutions, such as vector databases, typically rely on static embeddings or fixed weights, failing to adapt to users' varying preferences across different modalities. To address this, we present OneDB, a distributed framework for multi-metric similarity search that integrates data management with learning-based techniques. Unlike traditional systems, OneDB features three key algorithmic innovations: (i) an adaptive metric weight learning model based on lightweight contrastive learning, which infers implicit user preferences from limited query examples; (ii) a dual-layer indexing strategy that combines global partitioning with modality-aware local indexing to handle heterogeneous data distributions efficiently; and (iii) an end-to-end parameter tuning module leveraging deep reinforcement learning to optimize system performance in dynamic environments. Extensive experiments on real-world datasets demonstrate that OneDB captures user intent effectively, achieving 12.63%--30.75% higher accuracy and 2.5--5.75× faster retrieval speeds compared to state-of-the-art vector search systems.