TabQueryBench is a query-centric benchmark that uses SQL-shaped analytical queries as structural assessors for synthetic data fidelity, and provides an extensible foundation for query-centric synthetic-data evaluation.
Abstract
Synthetic tabular data support use cases like data sharing, model development under access restrictions, and rapid prototyping of analytical workflows. Modern generative models are evaluated by their statistical similarity, correlation structure, privacy, and downstream machine-learning utility. However, such evaluations leave a gap: they rarely test the structure that matters for analytical queries. We present TabQueryBench, a query-centric benchmark that uses SQL-shaped analytical queries as structural assessors for synthetic data fidelity. It provides an extensible foundation for query-centric synthetic-data evaluation. From 12 public sources of analytical queries, TabQueryBench taxonomizes recurring cross-domain logic into 44 reusable query templates and grounds them to each dataset via a policy-guided template-to-SQL pipeline. This makes queries schema-aware while preserving comparability across generative models. Across 49 datasets and 11 generative models, it activates 10-12 templates per dataset, producing more than 100 executable SQL queries per dataset. Our systematic experiments show five main patterns. First, current tabular generative models can have good distance-based fidelity, but they still fall short on query-centric fidelity: RealTabFormer achieves the highest query-centric fidelity, but it only reaches 0.75 +/- 0.15 (REAL data score is 1.00). Second, tabular generative models struggle with very high-cardinality discrete support. Third, SOTA generative models preserve good global conditional query-centric fidelity, but fail more on local queries. Fourth, tail fidelity deteriorates as queries move toward the extreme tail; even the best model recovers only about 40.7% of real rare values. Finally, there is a fidelity-cost tradeoff in tabular generation: BayesNet offers the strongest tradeoff, with slightly lower query-centric fidelity but much lower generation cost.
Hybrid queries—natural language questions over structured data that require both database capabilities and LLM reasoning—have recently emerged as a prominent research topic. However, existing solutions remain overly dependent on manual workflows, and current benchmarks are limited in scale and diversity. To bridge this gap, we present (1) HyQBench \xspace, a large-scale benchmark with 60\sim 90× more queries than prior work, built on 3× more databases; (2) AutoHyQ \xspace, an automated pipeline that can execute existing methods without manual intervention; (3) multi-dimensional, fine-grained evaluation metrics for comprehensive assessment. Through extensive experiments across multiple hybrid query approaches on diverse LLM backbones, we reveal their strengths and limitations, and identify research opportunities for advancing this emerging field. Our code and data are available at https://github.com/XMUDM/HyQBench.
Bo Li, Chenzhan Wang, Long-Kang Lin et al.· Proceedings of the 32nd ACM...· 0 citations
Slow queries frequently cause severe performance bottlenecks in database management systems. Diagnosing their root causes online risks exacerbating resource contention, while data privacy regulations often prohibit copying production data to test environments. Synthesizing a proxy database from non-intrusive metadata that induces the query optimizer to generate the same physical execution plans is therefore critical for offline diagnosis. High-fidelity reproduction requires preserving global statistical distributions while enforcing exact local cardinalities. Existing data-driven and workload-aware approaches cannot satisfy both requirements simultaneously. We present DBRepro, an automated end-to-end framework that formulates database generation as a constrained distribution synthesis problem. DBRepro initializes a global distribution from lightweight column statistics, extracts execution constraints from target queries, and progressively adjusts the distribution to satisfy these constraints while preserving the global distribution. Experiments on TPC-H and SSB show that DBRepro reduces cardinality error by up to 20.3% over a data-driven baseline while maintaining identical plan consistency. Compared with a workload-aware baseline, it reproduces 15% more consistent execution plans and reduces latency proportion error by 21.5%. We further validate DBRepro on a nearly 1 TB real-world dataset managed by KingbaseES, where it reproduces the execution performance of complex slow queries with high fidelity.
Zhao-Yang Zhang, Shuang Liu, Deng-Feng Xu et al.· 0 citations
This work recast five text-to-SQL datasets as retrieval tasks at both table and column granularity, covering realistic and enterprise-scale schemas under two document representations, and proposes corpus-adaptive fine-tuning, establishing schema linking as a standalone retrieval task and lightweight, label-free corpus adaptation as a practical route to deploying it at enterprise scale.
Qingcheng Zeng, Puxuan Yu, Aman Mehta et al.· arXiv.org· 0 citations
This work introduces ExpeSQL, a zero-shot, open-source–compatible, and efficient framework that combines divide-and-conquer reasoning, Best-of-N candidate selection, and self-critique with experience-guided refinement that establishes a new paradigm for deployable, self-improving Text-to-SQL systems in dynamic, real-world environments.
LLM-QOpt++ is presented, a novel hybrid, confidence-aware query optimization framework that unifies traditional CBO estimation, machine learning–based cost prediction, and large language model (LLM) reasoning within a single adaptive pipeline.