A Large Language Model "Cognitive Stethoscope" for Dementia Screening from Naturalistic Conversations in Primary Care

Abstract

Early detection of cognitive impairment in primary and long-term care remains a critical unmet need, as structured neuropsychological tests impose prohibitive time and psychological burdens. We evaluated a "Cognitive Stethoscope" framework — a multimodal Large Language Model (LLM) approach that screens for impairment by analyzing naturalistic clinical conversations without specialized equipment. In a prospective study of 106 participants (mean age 81.7±8.7 years; Normal n = 77, MCI n = 6, Dementia n = 23), we compared a multimodal pipeline (Gemini 2.5 Pro) processing raw audio directly against three text-based pipelines (Gemini 2.5 Pro, Claude Opus 4.6, GPT 5.5), evaluated against clinical diagnosis using sensitivity, specificity, NPV, and weighted Cohen’s kappa (κ); findings related to the MCI subsample (n = 6) are reported as strictly exploratory. GPT 5.5 matched Gemini 2.5 Pro (Text) for the highest binary sensitivity (82.8%) and NPV (93.2%), while Gemini 2.5 Pro (Multimodal) achieved the highest specificity (96.1%) and PPV (87.5%). Claude Opus 4.6 showed a balanced profile (sensitivity 75.9%, specificity 92.2%, κ = 0.602). All four models demonstrated comparable overall performance (weighted κ range: 0.539–0.602), reflecting a meaningfully different performance landscape compared with the original models. Qualitative analysis identified five discrepancy patterns: physical confounder halo effects, white-coat anxiety, facade reactions concealing decline, cross-sectional assessment limits, and somatic illness confounding. Multimodal LLM analysis of unstructured conversation provides accurate, non-invasive screening within routine care workflows. The high NPV enables confident dementia rule-out, and the model's resistance to facade reactions offers a practical adjunct to clinician judgment in primary and long-term care.