Sep 2026· Zenodo (CERN European Organization for Nuclear Research)
Abstract
Large language models (LLMs) could automate reasoning over structured scientific databases in chemistry and materials science, but this requires purpose-built agentic architectures, not simply better models. We present an agentic workflow for systematically querying and extracting data from the NIST Standard Reference Database 46 (SRD-46) to perform autonomous speciation and thermodynamic analysis, with three contributions: (1) a hierarchical toolset, served by Model Context Protocol (MCP) framework, which converts natural-language scientific questions into validated SQL over a database enriched with canonical chemical identifiers, equilibrium-reaction maps, and fingerprint-similarity matrices; (2) an induction-deduction-validation reasoning loop with compactors that preserve key chemical context across multi-turn sessions under dynamic memory limits within the same session; and (3) a soft time/turn-budget hook that maintains coherent reasoning within resource constraints. The resulting framework is LLM-agnostic and spans queries from direct lookups to cross-species comparisons to open-ended tasks like estimating unknown Fe complexation constants and selecting ligands for acidic Fe electrodeposition. We benchmark four frontier models (Claude Opus 4.6, Claude Sonnet 4.6, GPT-5, and GPT-5.4) on 49 prompts across four complexity levels, repeated five times (~1,000 answers), using automated claim-level validation and regex-based tool-result matching. All models retrieve reliably on direct-lookup tasks, confirming the architecture bridges natural-language reasoning and structured database access. Beyond lookups, behaviors of models diverge: GPT-5.4 converges rapidly with high per-claim verifiability but limited exploratory depth, whereas Opus 4.6 pursues broader, cross-examined reasoning at the cost of slower convergence and more unsupported claims. This reveals a fundamental tradeoff between convergent efficiency and exploratory breadth on fuzzy scientific queries that scalar benchmarks cannot resolve. It also suggests that fuzzy scientific queries require expert-stewarded reasoning, not faster convergence criteria or higher benchmark metrics alone. Our results demonstrate that domain-expert-designed agentic architectures and evaluation frameworks are essential infrastructure for AI-driven scientific discovery, and that mixed-model deployment strategies may be needed to balance efficiency and explanatory depth for different classes of scientific questions. Revision v1.0.0(R) documents a newly validated Python environment, packages the supplied large databases in lossless ordinary ZIPs with automatic verified local restoration, and updates installation and database documentation. The dependency lock was prepared on 21 September 2026; it does not reconstruct the historical experiment environment, and no benchmark results were regenerated.
The results are packaged in the Greenfield Startup Model (GSM), which explains the priority of startups to release the product as quickly as possible, and the need to shorten time-to-market, by speeding up the development through low-precision engineering activities.
Carmine Giardino, Nicolò Paternoster, M. Unterkalmsteiner et al.· IEEE Transactions on Softwar...· 178 citations· ⚡14
Software startup companies develop innovative, software-intensive products within limited timeframes and with few resources, searching for sustainable and scalable business models.
M. Unterkalmsteiner, P. Abrahamsson, Xiaofeng Wang et al.· e-Informatica Software Engin...· 157 citations· ⚡17
This study conducts a case survey study based on the secondary data of the major pivots happened in 49 software startups, and demonstrates that customer need pivot is the most common among all pivot types.
Sohaib Shahid Bajwa, Xiaofeng Wang, Anh Nguyen-Duc et al.· Empirical Software Engineeri...· 127 citations· ⚡15
The comparison of adopter and non-adopter sample reveals three potential adoption inhibitor, security, data privacy, and portability, which underlines the importance of the technical and security perspectives for research investigating the adoption of technology.
Nattakarn Phaphoom, Xiaofeng Wang, S. Samuel et al.· Journal of Systems and Softw...· 111 citations· ⚡8
The ongoing work building a Raspberry Pi cluster consisting of 300 nodes is presented, with potential use cases being an inexpensive and green test bed for cloud computing research and a robust and mobile data center for operating in adverse environments.
P. Abrahamsson, S. Helmer, Nattakarn Phaphoom et al.· IEEE International Conferenc...· 110 citations· ⚡7
The results indicate that software developers are a slightly happy population, but the need for limiting the unhappiness of developers remains, and 219 factors representing causes of unhappiness while developing software are identified.
D. Graziotin, Fabian Fagerholm, Xiaofeng Wang et al.· International Conference on...· 84 citations· ⚡6
Related blog posts
Microsoft Research Blog· microsoft.comOct 12, 2026
StudentSim is a research system for evaluating and improving AI tutoring tools before they are tested with learners. It models how individuals may engage with tutors, giving feedback to improve tutor performance across subjects like chess, writing, and mathematics. The post Simulating student learning for better AI tutoring appeared first on Microsoft Research.
MIT News · Artificial Intelligence· news.mit.eduOct 8, 2026