Sep 2026· Studies in Health Technology and Informatics· Vol 340, pp.
234-241
· 0 citations
Medicine
TL;DR
Five current large language models could reproduce expert-validated statistical analyses from a published clinical workflow across three statistical tasks: descriptive table generation, Kaplan-Meier survival analysis, and Cox proportional hazards modelling.
Abstract
Statistical analysis of clinical data requires expertise in medical statistics. Large language models (LLMs) are increasingly used for code generation and may support both descriptive and advanced analyses, but their reliability remains uncertain. This study evaluated whether five current LLMs (GPT 5.3, Claude Sonnet 4.6, Gemini 2.5 Flash, Perplexity, and Grok) could reproduce expert-validated statistical analyses from a published clinical workflow across three statistical tasks: descriptive table generation, Kaplan-Meier survival analysis, and Cox proportional hazards modelling. All models received identical datasets and standardised prompts. Their outputs were compared with analyses performed by two experts trained in mathematical statistics and evaluated for grouping correctness, numerical accuracy, missing-value handling, and quality of generated R code. All five models produced correct descriptive statistics once dataset variables were specified explicitly. Two models failed the initial descriptive benchmark because of variable-name ambiguity, but both recovered after prompt clarification. All models reproduced the correct Kaplan-Meier p-value, although figure completeness differed. In the Cox benchmark, four models reproduced all required hazard-ratio terms, while one omitted the interaction results. These findings suggest that LLMs may support clinical research workflows, but their outputs still require careful validation before use.
HeartVar demonstrates that an LLM constrained by a domain-specific prompt and grounded in structured evidence can produce variant interpretations of first-pass quality for a cardiovascular disease cohort; however, it is not intended to replace manual assessment by a qualified variant curator.
Jamie-Lee M. Thompson, Debjani Das, Sally L. Dunwoodie et al.· bioRxiv· 0 citations
Background Large language models (LLMs) are increasingly used to support statistical analyses in biomedical research. However, their ability to accurately and reproducibly compute descriptive statistics directly from datasets has received limited evaluation. Objective To compare the accuracy, within-modality repeatabil...
P. Sebo, Ting Wang· American Heart Journal Plus:...· 1 citation
PurposeLarge language models (LLMs) are increasingly used in clinical data workflows, but most clinicians lack the time, training, or support to build predictive models. We evaluated whether a ChatGPT-assisted workflow could enable clinicians to independently construct and execute a basic predictive model using routine...
J. Caron, O. Rodriguez, Leila Raden et al.· The American surgeon· 0 citations
BACKGROUND
National Surgical Quality Improvement Program (NSQIP) data collection depends on labor-intensive manual chart abstraction, limiting efficiency, increasing cost, and necessitating patient sampling. This study evaluated whether a large language model (LLM) could accurately abstract unstructured NSQIP breast re...
Tahera Alnaseri, Yasmine Ibrahim, Ansgar Grunseid et al.· Journal of the American Coll...· 0 citations
An LLM-as-a-judge framework for evaluating responses to patients’ periprocedural questions support its potential as a scalable assistive monitoring tool while preserving physician oversight for uncertain or safety-sensitive interactions.
B. Collaço, Nadia G. Wood, Yun-Guo Yu et al.· Bioengineering· 0 citations
Related blog posts
Microsoft Research Blog· microsoft.comNov 16, 2026
AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.
MIT scientists developed a training method to generate models that simulate the effects of human activities on climate much more quickly than other approaches.
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.