How users actually use LLM-powered data systems is examined: how they struggle to specify and validate queries, how they iterate extensively to converge on bespoke operators specific to their documents and domains, and what these patterns imply for the systems and benchmarks their community should be building.
This study provides an initial assessment on the feasibility of using LLMs as a KB for CH, using “Galois”, a recent framework for executing Structured Query Language (SQL) queries over LLMs with logical and physical optimizations tailored to the model’s behavior.
Querying LLMs as digital libraries is feasible, but its effectiveness depends on model strength, deployment conditions, dataset structure, and execution strategy, and Galois remains valuable when relational discipline and controlled query execution are required.
Mirco Cazzaro, G. Silvello· International Journal on Dig...· 0 citations
A mixed-method, between-subjects user study comparing SQL-LLM, a GPT-4o-backed NLIDB, with Snowflake, a traditional SQL analytics platform finds that a verification burden survives the switch to natural language, so an NLIDB that hides the generated SQL would remove the very step users relied on to trust the answer.
A large-scale benchmark with 60\sim 90× more queries than prior work, built on 3× more databases, an automated pipeline that can execute existing methods without manual intervention, and multi-dimensional, fine-grained evaluation metrics for comprehensive assessment.
Bo Li, Chenzhan Wang, Long-Kang Lin et al.· Proceedings of the 32nd ACM...· 0 citations
Quiksight is a web-based conversational data analysis system powered by Large Language Models (LLMs) that enables users to interact with datasets using natural language and demonstrates the feasibility of LLM-powered conversational interfaces for democratizing data analysis.
Wumi Ajayi, Ihuanyachi O. OGBONNA, Daniel I. OGHINAN· Global Journal of Engineerin...· 0 citations
This work analyzes query logs from MotherDuck, a cloud analytics platform built on DuckDB, to show that real-world SQL extends far beyond analytical queries, with users frequently employing dialect-specific functionality that LLMs struggle with.
Till Döhmen, Adithya Krishnan, Hamilton Ulmer et al.· The VLDB journal· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
