An agentic AI framework integrating LLMs with specialized deep learning tools for glaucoma detection from fundus photography was developed and validated, suggesting generalizability across backbones, and may signal a shift from monolithic models toward orchestrated multi-agent systems in medical AI.
Abstract
Large language models (LLMs) show promise in medical image interpretation but suffer from hallucination, limited accuracy, and run-to-run inconsistency. We developed and validated an agentic AI framework integrating LLMs with specialized deep learning tools for glaucoma detection from fundus photography. The workflow had three steps: (1) LLM initial assessment; (2) function calling to invoke specialized tools for image quality (QAModel, FundaQ-8), glaucoma classification (SwinV2-Tiny), and optic disc/cup segmentation (SegFormer-B0); and (3) LLM reflection integrating the initial impression with tool outputs. Two LLMs (Gemini 2.5 Flash, GPT-5.4 mini) were evaluated on two public datasets (ORIGA, n=100; RIM-ONE-v3, n=100) under uncropped and cropped fields of view; all images were independently graded by a masked fellowship-trained glaucoma specialist. The agentic workflow improved classification accuracy by 16 to 47 percentage points across all conditions, reaching within 6 points of the specialist; on RIM-ONE-v3 the best configurations matched the specialist accuracy of 88%. LLM-alone approaches failed in two ways: GPT-5.4 mini showed positive bias (sensitivity 95-100%, specificity 0-5%), while Gemini 2.5 Flash varied stochastically between runs; the agentic workflow corrected both. Cup-to-disc ratio error fell 15-50% (MAE 0.156-0.228 to 0.104-0.132), and correlation with specialist grading rose from weak (r=0.12-0.39) to moderate-strong (r=0.59-0.84). Run-to-run consistency rose from near-random (kappa as low as -0.01) to near-perfect (kappa up to 0.96). Integrating LLMs with specialized tools addressed key limitations of LLM-alone approaches, including over-diagnosis and run-to-run variability. Gains held for both LLMs, suggesting generalizability across backbones, and may signal a shift from monolithic models toward orchestrated multi-agent systems in medical AI.
FOCUS provides a practical framework and public benchmark to assess generalization, reliability, and robustness beyond single-dataset leaderboards and demonstrates that retinal model evaluation is inherently multidimensional.
David S. Restrepo, Chen-Wei Wu, L. Nakayama et al.· 0 citations
Diabetes mellitus (DM) is one of the most common causes of preventable blindness in the world, and DR is a major cause of working-age adults losing their vision. Mass screening programmes are necessary but limited by the availability of trained ophthalmologists, and the cost of such programmes. This paper introduces a...
Varsha Pandagre, N. Deshpande, Shilpa Gite· International Conference on...· 0 citations
ChatGPT demonstrated the strongest accuracy, justification-accuracy association, and confidence-accuracy correlation compared with Claude, Gemini, and Grok and showed a positive correlation between confidence and accuracy.
Lia Huo, Astha Chandra, Michael Balas et al.· Retina· 0 citations
Background/Objectives: Retinal fundus photography is the most widely available means of screening for sight-threatening disease, but it is difficult to read: several conditions leave overlapping signs on the same photograph, the features that separate them are often subtle and confined to small regions of the retina, a...
Foundation models for colour fundus photography (CFP) are candidate visual encoders for multimodal medical AI, but existing models are large and are assessed under fine-tuning rather than the frozen-encoder conditions such systems impose. We present RFM (Retinal Foundation Model), a 64-million-parameter Vision Transfor...
J. Fajtl, R. Welikala, R. Shakespeare et al.· medRxiv· 0 citations
PURPOSE
To develop and validate a deep learning model for simultaneous detection of nine fundus conditions from a single color fundus photograph.
METHODS
A development dataset of 236,242 color fundus images was assembled from 17 heterogeneous sources and partitioned at the patient level into training (70%), tuning (1...
Jungwoo Ha, Ye-On-Hee Choi, Dan-Bi Lee et al.· Asia - Pacific Journal of Op...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.