GPT-4 improves sex-specificity in cardiovascular patient education but may perpetuate gender biases: A mixed-methods audit

Abstract

Patient education materials for cardiovascular disease (CVD) prevention frequently omit clinically important differences in disease manifestation, risk, and prevention between sexes. Socially constructed gender norms further shape how health information is communicated and received. Large Language Models (LLMs) like GPT-4 offer a potential tool for personalizing health communication, but their capacity to incorporate evidence-based sex differences without introducing or perpetuating gender biases is unknown. We identified seven publicly available English-language CVD prevention handouts from major health organizations. Using the GPT-4 API in August 2025, we generated sex-specific revisions for a 55-year-old male and female audience via standardized prompts. We provided structured prompts instructing the model to include evidence-based, sex-specific risk factors and symptoms. Original and revised materials were evaluated using Flesch-Kincaid Reading Ease, a novel 10-point sex-inclusivity checklist, and qualitative thematic analysis using a reflexive approach informed by thematic analysis principles. GPT-4 revisions substantially improved sex-inclusivity scores (Original median: 3.0/10, IQR = 0.0–3.0; Male-tailored median: 7.0, IQR = 7.0–8.5; Female-tailored median: 10.0, IQR = 10.0–10.0). Readability was maintained. However, qualitative analysis identified instances where outputs introduced content inconsistent with an equity-oriented framework: female-tailored content occasionally framed female symptoms as deviations from a male norm, male-tailored content frequently omitted erectile dysfunction as a CVD risk marker despite explicit prompting, and both versions at times reflected gendered social stereotypes (e.g., “bottling up emotions”) rather than clinical evidence. LLMs can rapidly improve sex-specificity in patient education materials but may also reproduce gender stereotypes and linguistically biased framings present in their training data. Their integration into clinical content development requires critical human oversight to support, rather than undermine, health equity goals.