Skip to content
Review Open access

Using large language models to facilitate literature review and data extraction for infectious disease models: COVID-19 as a test case

Sep 2026 · medRxiv · 0 citations
Medicine

TL;DR

An open-source, end-to-end pipeline is built to simulate LLM performance in a hypothetical scenario where they were available to inform COVID-19 models developed during the first four months of 2020, and shows that current models can extract transmission parameters from unstructured literature accurately enough to inform outbreak modelling.

Abstract

Infectious disease transmission models are governed by parameters informed by systematic review of epidemiological literature. Large language models (LLMs) could facilitate this, but the reliability of the results to inform models has not been tested. We built an open-source, end-to-end pipeline to simulate LLM performance in a hypothetical scenario where they were available to inform COVID-19 models developed during the first four months of 2020. It screened articles and extracted the reproduction number, serial interval, and incubation period from full-text PDFs. We applied it to 2,067 PubMed/medRxiv records published 31 December 2019-30 April 2020 using four models (GPT-5-mini, GPT-5.4, Claude Opus 4.8 and Gemini 2.5 Pro) and evaluated it against full-corpus human screening and 50-article extraction gold standards. We combined models post hoc, pooled the extracted values into an infectious disease (SEIR) model, and ran sensitivity analyses to test approaches to improve extraction accuracy. Screening sensitivity was 0.72-0.95 and specificity 0.92-0.99. For articles reporting few values, extraction F1 was 0.90-0.96 with precision 0.91-1.00; across all articles, including those with dozens of stratified estimates, recall fell to 0.38-0.78. No fabricated values observed; errors were misassignments of values filed under the wrong parameter, or borrowed values treated as the study's own. Incomplete extraction from dense articles was mainly due to prompting and output format, not model capability. Ensembling allowed recall-precision trade-offs, and correctness increased with model agreement, from about 30% at one vote to 92-95% at four. The pipeline processed the corpus in hours versus an estimated 130-265 person-hours of manual effort. Our results show that current models can extract transmission parameters from unstructured literature accurately enough to inform outbreak modelling. The bottleneck lies in task specification, and careful prompt and output schema design are key to reducing misassignment errors. Human effort is best directed at workflow development and provenance validation.

Read PDF

Similar papers

Open access Aug 2026

Benchmarking commercial large language models for gene-disease-phenotype extraction from full-text human genetics literature

Manual curation of gene-disease-phenotype relationships from the human genetics literature is a persistent bottleneck for maintaining its bioinformatics databases. Whereas large language models (LLMs) offer a promising alternative, there is currently no systematic benchmark that evaluates whether state-of-the-art comme...

Dan-Qing Yin, M. Leung, Darren Wan Ho Pun et al. · 0 citations
Open access Sep 2026

LLM-enabled Natural History Study Analysis to Support Rare Disease Research

Background: Rare diseases affect an estimated 300 million people worldwide, yet the research needed to guide diagnosis and treatment is often fragmented across multiple unstructured literature sources. Natural history studies (NHS) are a key source of this evidence, but manually extracting structured information from N...

Kevin Li, E. Sid, Qian Zhu · 0 citations
#large language models Review Open access Sep 2026

Large Language Models for Clinical Note Simplification: A Systematic Review and Experimental Evaluation of Medical Text Readability.

The findings suggest that conventional readability metrics should be extended with domain-specific measures to more accurately assess comprehensibility in medical texts and that large Language Models show strong potential to enhance the accessibility of clinical documentation for patients.

M. Teichmann, Pelin Özkara Menekseoglu, Julian Schwarz et al. · 0 citations
Review Open access Sep 2026

Benchmarking five large language models in medical genetics: a bilingual comparative evaluation using published and novel expert-authored questions

In an exploratory analysis, residual errors showed concentrated and concordant patterns, clustering in multistep Bayesian reasoning and evolving facts, suggesting that a second model may provide limited independent protection on such items.

Ö. Gündoğdu Öğütlü, Benjamin D. Solomon, Y. Çelik · 0 citations
Review Open access Sep 2026

Citation reliability of frontier large language models in medical writing and its automated verification

Large language models are increasingly used to draft medical manuscripts, yet their citations are unreliable and clinicians lack a validated way to verify them, so LLM-generated citations require identifier-level verification, and CoVe provides it at expert-level accuracy.

R. Shin, J.-M. Lee, J. Park et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.