A mixed-method, between-subjects user study comparing SQL-LLM, a GPT-4o-backed NLIDB, with Snowflake, a traditional SQL analytics platform finds that a verification burden survives the switch to natural language, so an NLIDB that hides the generated SQL would remove the very step users relied on to trust the answer.
The study empirically demonstrates comparative behaviours of the current LLMs, and suggests the use of execution-aware evaluation in the studies and deployment of Text-to-SQL.
Bharath Kumar N, T L Manasa· International journal of com...· 0 citations
This paper explores the practical implementation of a Chatbot designed to connect natural language communication with database interactions by generating Structured Query Language (SQL) queries. Utilizing the OpenAPI framework, a widely-used specification for building APIs, our approach aims to enhance the flexibility and interoperability of the ChatBot. The system not only interprets user queries expressed in natural language but also transforms them into syntactically correct and semantically meaningful SQL commands. This paper provides an overview of the current landscape of natural language processing (NLP) Chatbots, discusses the architecture and design considerations of our implementation, and highlights the methodology for training and fine-tuning using state-of-the-art NLP models. Real-world use cases are presented to showcase the practical applicability of the Chatbot, demonstrating its ability to handle various complexities inherent in natural language queries. This work contributes to the field by presenting a scalable and robust solution that combines NLP and database interactions through the utilization of OpenAPI, with potential implications for improving human-computer interactions within database systems.
R. Santosa, Hartantya Ainiyatuts Tsaniyah, Yoga Ari Tofan et al.· IPTEK: The Journal for Techn...· 0 citations
How users actually use LLM-powered data systems is examined: how they struggle to specify and validate queries, how they iterate extensively to converge on bespoke operators specific to their documents and domains, and what these patterns imply for the systems and benchmarks their community should be building.
Shreya Shankar, Aditya G. Parameswaran, U. Berkeley· 0 citations
It is suggested that structured prompt engineering provides a practical alternative to model fine-tuning for locally deployed LLMs, offering an effective balance between SQL generation accuracy, computational efficiency, and data privacy.
Nurjayanti Nurjayanti, A. Adiwijaya, A. Romadhony et al.· Jurnal RESTI (Rekayasa Siste...· 0 citations
BIRD-History is introduced, a benchmark consisting of 1,393 tasks across 11 databases, designed to evaluate text-to-SQL systems'ability to ground underspecified natural language questions using historical SQL scripts, and a plug-in retriever that extracts five types of external knowledge from historical SQL scripts, then retrieves and reranks relevant fragments for query generation.
Yunfan Zhou, Qiming Shi, Yi-Zhou Yang et al.· 0 citations
This work shows that introspection and sampling are complementary but not disjoint on BIRD-Interact-Lite, a Text-to-SQL benchmark with 300 tasks, annotated ambiguities, and an LLM-based user simulator, and proposes a proposed multi-label grammar that substantially narrows it without affecting downstream context.
Leonhard Liu, Patrick K. Erdelt, Two· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
