Profile-associated financial and access-related framing in LLM-generated pediatric asthma referral plans: a factorial audit of seven large language models

Abstract

Background Large language models (LLMs) are increasingly considered for clinical documentation, referral support, and patient-facing communication. Biomedical accuracy alone may be insufficient for safe deployment if generated plans vary in access navigation, financial-access language, referral specificity, or tone across socially meaningful patient cues. Objective To evaluate whether onomastic and bundled geographic-access signals are associated with differences in LLM-generated pediatric asthma referral plans. Methods We conducted a cross-sectional 2 × 2 factorial audit of seven commercial LLMs. A standardized vignette described a 5-year-old boy with moderate persistent asthma, persistent nocturnal symptoms, FEV1 of 70% predicted, and an Asthma Control Test score of 16. Patient name, Liam Miller versus DeShawn Washington, and address/geography, Palo Alto, CA versus Indianola, MS, were manipulated while clinical facts were held constant. Each model generated 20 responses per profile, yielding 560 referral plans. Outputs were scored using a prespecified Automated Structural Competence Scoring framework. The primary endpoint was response-length-adjusted M16 Financial-Access Term Rate, analyzed using a negative-binomial model with log word-count offset and LLM fixed effects. Key secondary endpoints were controlled using Benjamini-Hochberg false-discovery-rate correction. Results In the response-level primary model, the DeShawn name signal was associated with a higher financial-access term rate (IRR, 1.47; 95% CI, 1.23–1.77; p < 0.001), as was the bundled geographic-access signal (IRR, 2.40; 95% CI, 2.02–2.85; p < 0.001). The name-signal association was directionally similar but less precise in model-profile aggregated sensitivity analysis. The interaction term was below 1.0 (IRR, 0.79; 95% CI, 0.63–1.00; p = 0.048), indicating no positive multiplicative synergy. Institutional Specificity and Triage Ranking were at ceiling. SDOH Recognition Depth, Location-Friction Acknowledgment, Navigator Recommendation, and Empathy/Subjectivity differed by profile, whereas Access Priority remained low and non-significant after correction. Human validation showed moderate endpoint-specific reliability. Conclusions LLM-generated pediatric asthma referral plans varied in financial-access, geographic-access, navigation, SDOH-recognition, and selected tone-related framing. These findings do not establish discriminatory intent, clinical equivalence, downstream harm, or positive synergistic interaction, but support evaluating structural and access-related framing alongside biomedical content in clinical LLM audits.