Comparative Diagnostic Accuracy of Large Language Models for STEMI Classification

Abstract

Background: Early and accurate identification of ST-Elevation Myocardial Infarction (STEMI) in the prehospital setting would reduce morbidity and mortality (Rao et al. 2025). Artificial Intelligence (AI) tools may assist emergency medical personnel by providing rapid electrocardiogram (ECG) interpretation and differential diagnosis (Chen et al. 2022; Nallamothu et al., 2015). Objective: To evaluate the diagnostic accuracy of ChatGPT-4.0, Gemini 2.5 Pro, and a hybrid model combining ECG-GPT + ChatGPT-4.0 in classifying STEMI versus non-STEMI cases based on ECG and clinical data inputs. Methods: Fifty-six consecutive de-identified cases (28 STEMI, 28 non-STEMI) from Staten Island University Hospital EMR (1/2025-6/2025) were analyzed. Each case included demographics, vital signs, chief complaints, and a representative 12-lead ECG. The three AI models were independently asked to classify each case as STEMI or not STEMI. Results: For the STEMI cohort (n=28), detection rates were: ChatGPT-4.0 21.4%, Gemini 2.5 Pro 67.9%, and ECG-GPT + ChatGPT hybrid 71.4%. For the non-STEMI cohort (n=28), correct classification rates were: ChatGPT-4.0 92.9%, Gemini 2.5 Pro 28.6%, and ECG-GPT + ChatGPT hybrid: 96.4%. The ECG-GPT + ChatGPT hybrid model demonstrated the highest diagnostic accuracy at 83.9%. Conclusion: While ChatGPT-4.0 showed high specificity, it lacked sensitivity for STEMI detection. Gemini 2.5 Pro improved sensitivity but yielded higher false positives. The hybrid ECG-GPT + ChatGPT model outperformed both standalone models, suggesting that multimodal AI integration may enhance triage accuracy for prehospital care. Despite the small population size and limitations associated with this, these results propose a viable means to hypothesize that AI may have potential in assisting responders with early STEMI identification. Future improvements in model precision could support the implementation of AI-based triage systems in emergency response settings, reducing time to treatment and improving outcomes by aiding direct cardiac catheterization lab transfers without unnecessary delays (Garvey et al., 2012; Squire et al., 2014).