Hollywood is introduced, a synthetic IMDb-compatible benchmark generator that combines LLM-generated semantic dictionaries with deterministic temporal-graph-based relational data generation and demonstrates that Hollywood induces cardinality estimation errors comparable to or exceeding those observed on the original IMDb dataset.
Abstract
The IMDb real-world dataset of the JOB benchmark has been extensively used in the last decade as part of the research line on cardinality estimation, given its ability to stress test both traditional and learned estimators. However, unlike the synthetic TPC family, it does not come with a scale factor, being a simple dump. We introduce Hollywood, a synthetic IMDb-compatible benchmark generator that combines LLM-generated semantic dictionaries with deterministic temporal-graph-based relational data generation. We analyze a preliminary Hollywood-200K, which contains 200,000 primary movies, generated series and episode title rows, 19.7M IMDb-style rows, and 213 nonzero JOB-Light, JOB, and JOB-Complex queries. Experiments with two open systems demonstrate that Hollywood induces cardinality estimation errors comparable to or exceeding those observed on the original IMDb dataset. The release includes generation settings and prompt/LLM-output provenance together with adapted SQL and labels, enabling tests of whether cardinality estimators generalize beyond a fixed movie snapshot and distribution.
Querying LLMs as digital libraries is feasible, but its effectiveness depends on model strength, deployment conditions, dataset structure, and execution strategy, and Galois remains valuable when relational discipline and controlled query execution are required.
Mirco Cazzaro, G. Silvello· International Journal on Dig...· 0 citations
This work recast five text-to-SQL datasets as retrieval tasks at both table and column granularity, covering realistic and enterprise-scale schemas under two document representations, and proposes corpus-adaptive fine-tuning, establishing schema linking as a standalone retrieval task and lightweight, label-free corpus...
Qingcheng Zeng, Puxuan Yu, Aman Mehta et al.· arXiv.org· 0 citations
To overcome scarce labeled data, Matryoshka Representation Learning truncation to 256 dimensions preserves 99.9% of retrieval quality while reducing dense-index storage and exact similarity arithmetic by a factor of three.
Konstantin Chesnokov, Chingiz Mingazov· 0 citations
TabQueryBench is a query-centric benchmark that uses SQL-shaped analytical queries as structural assessors for synthetic data fidelity, and provides an extensible foundation for query-centric synthetic-data evaluation.
This work revisits generation-target IRs under the BIRD regime and proposes HyMIR, a typed JSON IR designed for enterprise-scale schemas, indicating comparable representational coverage across the two benchmarks and quantifying a hard upper bound on attainable model performance under this IR and compiler stack.
A large-scale benchmark with 60\sim 90× more queries than prior work, built on 3× more databases, an automated pipeline that can execute existing methods without manual intervention, and multi-dimensional, fine-grained evaluation metrics for comprehensive assessment.
Bo Li, Chenzhan Wang, Long-Kang Lin et al.· Proceedings of the 32nd ACM...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.