Jun 2026· arXiv.org· Vol abs/2606.28601· 0 citations· 50 references
Computer Science
TL;DR
DBCC is a database-side middleware that performs offline structural and semantic compression together with lightweight online evidence purification and is model-agnostic and can be integrated into existing Text-to-SQL pipelines.
Abstract
Recent progress in Text-to-SQL has been driven by stronger language models and prompting strategies, yet performance on real enterprise benchmarks such as Spider 2.0 and BIRD remains far below that on classical academic datasets. We argue that the main bottleneck is no longer reasoning, but database representation. Real databases contain repeated audit columns, large groups of similar tables, opaque identifiers whose meanings are stored only in documentation, and extensive data dictionaries with little query-relevant information. Existing query-aware methods, including schema linking and retrieval-based schema selection, filter this raw context but still operate on redundant and verbose representations. We reformulate the problem as database context compression, a query-agnostic transformation that rewrites schemas, semantic descriptions, and external documentation into a compact representation. We formalize this transformation with the SGCF (Support-Gain Component Factorization) principle, which unifies repeated column extraction, isomorphic table templating, semantic componentization, and evidence purification under a single coverage objective. Based on SGCF, we propose DBCC, a database-side middleware that performs offline structural and semantic compression together with lightweight online evidence purification. DBCC is model-agnostic and can be integrated into existing Text-to-SQL pipelines. On Spider 2.0-Snow and BIRD, DBCC reduces input context by up to two orders of magnitude (from 2.6M to 34.7K tokens on the largest Spider 2.0-Snow subset), improves schema-linking strict recall from 0% to 56.5% under DeepSeek-V3.2 (63.1% under Claude Opus 4.7), and consistently increases end-to-end execution accuracy by 1.8-1.9% over three recent Text-to-SQL systems. Our code is open-sourced at https://github.com/MrBlankness/SchemaCompression.
This work recast five text-to-SQL datasets as retrieval tasks at both table and column granularity, covering realistic and enterprise-scale schemas under two document representations, and proposes corpus-adaptive fine-tuning, establishing schema linking as a standalone retrieval task and lightweight, label-free corpus adaptation as a practical route to deploying it at enterprise scale.
Qingcheng Zeng, Puxuan Yu, Aman Mehta et al.· 0 citations
Traditional Text-to-SQL research and benchmarks assume a known target database, overlooking settings in which a query must be routed within a large, heterogeneous database collection. We therefore study schema linking in a multi-database setting, where the system must first locate the target database and then construct a compact, SQL-relevant schema for generation. We propose MDB-Link, a hierarchical schema-linking framework that retrieves question-relevant columns from a global index, aggregates retrieval evidence to shortlist databases, and uses a budget-aware large language model (LLM) for database reranking, table selection, and column grounding. With Qwen2.5-14B, MDB-Link outperforms LinkAlign on MMQA, Spider2-Snow, and BIRD-dev in database localization and column selection while producing schema subsets close in size to the gold schemas. Exact match improves from 16.88 to 51.41 on MMQA, 2.50 to 9.17 on Spider2-Snow, and 12.52 to 38.01 on BIRD-dev. MDB-Link also runs faster than LinkAlign and AutoLink, demonstrating the effectiveness of hierarchical schema reduction for downstream SQL generation.
Beiyu Xu, Zhenyu Wu, Jiaoyan Chen et al.· 0 citations
Relational Database Management Systems (RDBMS) hold the vast majority of structured enterprise data. However, applying Large Language Models (LLMs) to perform natural language querying and tabular reasoning directly over relational schemas presents major bottlenecks: full schema injection quickly exhausts context windows, complex join relationships lead to hallucinated SQL syntax, and security risks like invalid query execution constrain enterprise adoption. To overcome these limitations, we present Schema-Aware Query Translation and Tabular Reasoning for Enterprise Databases aka Inference-from-RDBMS, an open-source framework designed for schema-aware query translation, dynamic context pruning, and execution-guided tabular inference over complex RDBMS structures. The framework extracts database metadata to construct an interactive Schema Relationship Graph (SRG), prunes unreferenced tables and foreign-key joins using semantic distance metrics, and passes a minimal schema sub-graph to an execution-validated text-to-SQL generator. We evaluate Inference-from-RDBMS across standard enterprise benchmarks (including SPIDER and real-world multi-table schemas). The experimental results demonstrate that our framework achieves an 88.4% execution accuracy, reduces prompt token overhead by 52%, and cuts end-to-end query translation latency by 41% compared to standard schema-injected baseline models. These results position Inference-from-RDBMS as a robust, secure, and cost-efficient solution for natural language database interfaces in large-scale AI applications.
Harshil Lodhiya· International Journal of Res...· 0 citations
Practical AI systems increasingly need to turn long, heterogeneous documents into queryable relational databases, not isolated spreadsheets. In domains such as finance, healthcare, education, transportation, and enterprise operations, downstream workflows rely on normalized schemas, entity identities, keys, cross-table relationships, and integrity constraints for analytics, compliance, auditing, and SQL-backed decision making. Existing Document-to-Table benchmarks are insufficient for this setting: flattening evidence into single tables can duplicate entities, obscure many-to-many relationships, create sparse records, and avoid testing whether extracted facts form a valid database instance. This creates an urgent need to evaluate document understanding as database construction rather than field extraction. We introduce Doc2DB-Bench, a benchmark for Document-to-Database construction, containing 203 long-document instances across 42 schemas and seven domain groups, with 117 entity tables, 132 relationship tables, 7,341 rows, and 41,935 cells. Built through a controllable DB-to-Doc synthesis pipeline and organized by a taxonomy of intra-table extraction and inter-table reasoning, the generated documents undergo authenticity verification, proving indistinguishable from real-world references. Doc2DB-Bench thus provides a testbed for reliable, auditable, and relationally faithful LLM-based data systems. The benchmark is publicly available at https://github.com/SetonLiang/Doc2DB-Bench.
Zhuowen Liang, Zhengxuan Zhang, Jiayang Wang et al.· 0 citations
Large language models (LLMs) achieve impressive accuracy on standard Text-to-SQL benchmarks such as Spider and BIRD, yet enterprise databases—with hundreds of tables and complex foreign key graphs—remain a practical bottleneck. We hypothesise that a single, measurable property drives most of this gap: the join-hop depth ( h ) of the query, defined as the number of foreign key edges that must be traversed to gather all required columns. We introduce the Join-Hop Depth (JHD) benchmark , 410 human-annotated questions stratified by h ∈ { 1 , . . . , 6 } over 12 enterprise-scale schemas. Experiments on five frontier LLMs confirm a sharp accuracy cliff: all models exceed 80% at h =1 but fall below 40% at h =4 and below 25% at h =6 —the typical depth of real enterprise analytics queries. To address this, we propose S CHEMA S COPE , a decomposition framework that partitions deep queries into a sequence of sub-queries with h ≤ 2 , executes them independently, and merges the re-sults. S CHEMA S COPE raises average execution accuracy from 46.8% to 67.3% on JHD (GPT-4o, h ≥ 3 ) and improves execution accuracy by +9 . 3 pp on the BIRD development set. Error analysis shows that decomposition eliminates wrong join path errors—the dominant failure mode at high h —and shifts the residual error budget toward condition and aggregation mistakes that are amenable to existing post-processing methods
Kaustubh S. Bukkapatnam, R. Malik· Proceedings of the First Wor...· 0 citations
BIRD is a recent enterprise-scale Text-to-SQL benchmark that exposes a regime not covered by earlier datasets such as Spider: it consists of 95 real-world databases, which on average contain seven tables each, with abbreviated column names, dense foreign key graphs, and noisy domain values. Generation-target intermediate representations (IRs) such as SemQL and NatSQL, which reduce the model’s output space to a structured plan and rely on a deterministic compiler to reconstruct full SQL, were developed before BIRD and evaluated mainly on Spider-like schemas. As a result, most subsequent BIRD-era work on Text-to-SQL has shifted toward prompt and pipeline engineering, while leaving the choice of generation target—raw SQL versus IR—largely unexamined for large, messy enterprise databases. In this work, we revisit generation-target IRs under the BIRD regime and propose HyMIR, a typed JSON IR designed for enterprise-scale schemas. HyMIR removes high-error structural clauses—JOIN, GROUP BY, and the SELECT projection—from the model’s output and delegates their reconstruction to a compiler that infers join paths from the foreign key graph, while a predicate passthrough channel carries the long tail of fragments that resist abstraction, including LIKE predicates, range bounds, nested IN subqueries, and dialect-specific date functions. To isolate the effect of the IR itself, we compare HyMIR against standard SQL as the generation target under strictly matched conditions: identical training data, backbone (Qwen-2.5-Coder-7B-Instruct), fine-tuning recipe (DoRA), and sampling protocol. On the BIRD development set, HyMIR improves execution accuracy from 52.50% to 55.48% under raw decoding and from 56.81% to 58.44% with $N{=}11$ result-set self-consistency. Across model scales (1.5B, 3B, 7B), the gains exhibit a mid-scale sweet spot—largest at 3B, with improvements of roughly + 11 points, and smaller but consistently positive at 7B—such that a 3B HyMIR model approaches the performance of a 7B SQL-tuned model while using fewer than half the parameters. Finally, a compiler-only oracle round-trip, which encodes gold SQL into HyMIR and compiles back, achieves 92.83% execution accuracy on BIRD and 91.0% on Spider, indicating comparable representational coverage across the two benchmarks and quantifying a hard upper bound on attainable model performance under this IR and compiler stack.