Scrydb, a Python library that enables lexical, semantic, and hybrid search within SQLite, is introduced, providing a lightweight solution for downstream tasks in information retrieval (IR) or agentic search.
Abstract
This work introduces scrydb, a Python library that enables lexical, semantic, and hybrid search within SQLite. For lexical search, scrydb leverages SQLite's full-text search extension FTS5. Semantic search builds on sqlite-vec, a SQLite extension for vector search. Furthermore, the library allows users to rerank and fuse retrieval results to combine both lexical and semantic approaches, providing a lightweight solution for downstream tasks in information retrieval (IR) or agentic search. We evaluate scrydb on various IR benchmark datasets and demonstrate its effectiveness in text retrieval based on keyword matching, semantic similarity, and rank fusion. In addition, we provide insights into query latency and the trade-off between efficiency and effectiveness. scrydb is available under the MIT license.
A hybrid search engine whose coreretrieval, ranking, and fusion components, an inverted index,TF-IDF, Okapi BM25, dense vector search, and score fusion, are implemented directly in Python rather than delegated to anexisting search or vector-database framework, so that the tradeoff can be measured directly.
Sabnam Ghimire· Proceedings of International...· 0 citations
This article explores asymmetric extractive semantic search, using the large language model (LLM) SBERT. It first examines traditional exact, keyword, and fuzzy match search algorithms, discussing their features and limitations. It then explores how semantic search leverages the capacity of an LLM to capture query inte...
Relational database queries depend on SQL and prior knowledge of the schema. This work proposes SABRE, a plain-text search architecture for relational databases based on the vectorization of tuples and queries and on retrieval by semantic similarity. The evaluation used a pilot study for selecting the embedding model a...
Mikael Sales, M. Duarte, Wellington Franco et al.· Brazilian Symposium on Datab...· 0 citations
Hybrid lexical-semantic retrieval combines signals derived from explicit term matching and dense semantic representations. However, an aggregate retrieval score can conceal heterogeneous behavior across information needs. This study examines that heterogeneity at query and predefined query-type levels using a controlle...
Mamadou Bakouan, Tiémoman Koné, Ko-Nan Yao et al.· International Journal of Adv...· 0 citations
Long-document retrieval-augmented generation (RAG) systems often apply a fixed retrieval strategy to different query types, although fact lookup, summarization, comparison, and multi-hop reasoning require different evidence structures. This paper presents HEC-RAG, a semantic-type routing framework for long-document RAG...
Wen-Kai Chen· Symposium on Designing Inter...· 0 citations
iPDB is demonstrated, a system that supports in-database LLM inference using an extended declarative SQL syntax and new optimizations that result in efficient query processing of LLM-enabled SQL queries that outperform state-of-the-art systems.
Udesh Kumarasinghe, Tyler Liu, Ahmed R. Mahmood et al.· Proceedings of the VLDB Endo...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
