An agentic framework that leverages Large Language Models (LLMs) for generating UDF-centric queries from natural language task descriptions in the medical domain is presented, demonstrating that structured tool orchestration with verification loops substantially improves generation quality.
The deployment of natural-language-to-SQL (NL-to-SQL) systems in primary healthcare requires more than accurate query generation: it also requires governed data access, robustness to local terminology, and reliable handling of ambiguous user requests. This study evaluated a pilot proof-of-concept integrating a Spanish-language NL-to-SQL assistant with a governed, read-only outpatient scheduling repository derived from the Rayen information system used in a Centro de Salud Familiar (CESFAM) setting in Renca, Chile. The data used by the prototype were accessed through an external company responsible for data management in this context. The prototype was implemented with MindsDB as an artificial intelligence (AI)-enabled database layer and operated on anonymized, delayed secondary scheduling data. Evaluation was conducted through a Slack interface using 252 audited interactions from 42 users, with six assigned interactions per user and up to three exchanges per interaction. SQL correctness reached 240/252 (95.2%), whereas both query correctness and answer correctness reached 144/252 (57.1%). These findings suggest that governed pilot deployment for outpatient schedule monitoring may be feasible under controlled institutional conditions, while indicating that the main remaining barriers are semantic rather than purely syntactic, specifically ambiguity handling, institution-specific operational language, and faithful answer verbalization. The study therefore contributes deployment-oriented pilot evidence and clarifies where operational Spanish NL-to-SQL remains fragile under real institutional constraints.
Isaac Daroch, Matías Rojas Cabrera, Rodrigo Muñoz Andrade et al.· Big Data and Cognitive Compu...· 0 citations
Clinical question answering over electronic health records (EHRs) increasingly relies on large language model (LLM) agents that retrieve structured patient data through external tools. Published benchmarks, however, evaluate these systems at a single patient-population size, and rarely measure the effect of backend representation from that of the retrieval interface design. This paper compares six retrieval configurations that vary along two axes: backend (a property graph database, a relational database and a dense vector index) and interface design (curated domain-specific tool calls, model-generated queries, full-text search, and single-shot dense retrieval). The evaluation covers a 334-question bank spanning six categories (simple lookup, multi-hop, temporal, cohort, reasoning, and unanswerable), instantiated at three nested population scales: 200, 2000, and 20,000 alive patients from a single Synthea cohort. Four models are compared: Claude Haiku 4.5, Qwen 2.5 72B, Llama 3.1 8B, and Llama 3.3 70B, spanning closed-frontier and open-source alternatives. Curated tool-calling configurations improve accuracy over retrieval-augmented baselines for capable models, but reduce accuracy for a small open-source model due to function-calling protocol failures. We report how accuracy, latency, and cost evolve with each approach, model size, and cohort size, supported by paired statistical tests and confidence intervals. All benchmark components, databases, and evaluation code are publicly available.
Leonidas Anagnou, Andreas Vezakis, Ioannis A Vezakis et al.· Future Internet· 0 citations
The empirical findings indicate that factorizing ML-aware SQL generation into four distinct stages—query routing, structured intent extraction, model or function selection, and template-guided SQL synthesis—enhances semantic controllability and token efficiency when formulating predictive natural language queries over database-native ML systems.
The results support a trade-off interpretation rather than a universal ranking: additional structural constraints changed failure modes and efficiency, but did not monotonically improve correctness or solve ambiguity and multi-turn state consistency.
Researchers need to answer ad-hoc questions about the contents of domain-specific archives but often lack the expertise to write structured queries on the metadata. We show that when domain vocabulary and semantics are captured in a well-designed Web Ontology Language (OWL) ontology, Large Language Models (LLMs) can generate accurate structured queries zero-shot, without task-specific fine-tuning, retrieval augmentation, or multi-agent orchestration. We present the Natural Language Knowledge Graph Query (NLKGQ) system, a framework and development process that enables natural language access to metadata in such archives. The framework includes a web interface that helps researchers pose natural language questions, which a domain-agnostic harness translates to SPARQL via an LLM and executes against a knowledge graph. The development process begins with capturing domain vocabulary and semantics in a formal OWL ontology. Domain-specific code then extracts metadata from archive sources and imports it into a knowledge graph defined by the ontology. Both are designed for reuse across domains. We demonstrate the system on metadata derived from a large-scale neuroimaging research archive, evaluating multiple LLMs and ontology representations. The best configurations achieve 100% accuracy on a 21-question competency and regression test set developed with domain experts. An ablation study across eight ontology representations reveals that readable entity names and semantic annotations are the dominant factors in accuracy, more significant than model choice or prompt engineering. We also compare SPARQL to an auto-generated SQL database as query backends, showing that OWL's structural features provide a substantial advantage over SQL DDL for LLM-driven query generation. Our demonstration domain requires local LLMs on modest institutional hardware to address privacy concerns for human subject data.
This paper studies the VKG-QA task, which enables users to interact with the VKGs through a natural language (NL) interface by translating their questions into SPARQL queries, and pro-poses NaVQA (Navigation-based VKG Question Answering), a framework leveraging Large Language Models.
Guohui Xiao, Haohan Xue, Lin Ren et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
