Stakeholder role-prompting fundamentally alters clinical decisions and ethical value frameworks of frontier LLMs, with the insurer role producing systematic denial of physician-endorsed, patient-preferred treatments.
Abstract
Background: Large language models (LLMs) are increasingly deployed in healthcare, where they may adopt different stakeholder perspectives, yet the effect of role-prompting on clinical ethical reasoning remains poorly characterized. Methods: We evaluated three frontier LLMs: Claude Opus 4.6, GPT-5.4, and Gemini 3.1 Pro across 25 ethically complex medical cases. Each model responded from three stakeholder perspectives (physician, patient, insurer) across three independent runs (675 total responses). Decisions were benchmarked against a six-physician panel. Ethical value prioritization was analyzed from physician- and LLM-provided ranked values. A Patient-Centric Decision Index (PCDI) was developed to quantify LLM decision alignment with patient-preferred out-comes. Results: Among 20 cases with clear physician consensus, LLMs prompted as an insurer reduced alignment with physician majority by 50% for GPT-5.4 (p = 0.004), 45% for Gemini 3.1 Pro (p = 0.008) and 10.5% (NS) for Opus 4.6. The insurer role shifted primary ethical values from beneficence (27%) to financial stewardship (20%) across all LLMs. Conclusions: Stakeholder role-prompting fundamentally alters clinical decisions and ethical value frameworks of frontier LLMs, with the insurer role producing systematic denial of physician-endorsed, patient-preferred treatments. These findings raise the need for standardized LLM patient-centricity benchmarks, and physician oversight when LLMs are deployed in clinical decision-making.
It is found that LLM access enhances performance on standardized clinical vignettes in all three countries, and policymakers should prioritize structured integration of LLMs as decision-support tools, combined with targeted training, local validation, and safeguards against automation bias rather than relying on access...
N. Rounding, L. S. Arif, Janine Berg et al.· 0 citations
Large Language Models (LLMs) achieve strong results on many medical benchmarks, but their clinical reasoning remains difficult to evaluate reliably. A central risk is an evaluation illusion: fluent and well-structured explanations can appear clinically convincing even when the final diagnosis is incorrect. We introduce...
Abin Roy, Afthab Salam Kanniyan, Jawadh Abdul Kabeer et al.· arXiv.org· 0 citations
Multiple-choice medical benchmarks are increasingly saturated, and recent rubric-based evaluations such as HealthBench have shown that open-ended clinical performance is far from solved - its"Hard"subset top score remains 32%. We present a small, deliberately difficult evaluation dataset of five clinician-authored clin...
Samiha A. Ismail, Fan X. Chen, Ali Merali· 0 citations
LLMs hold substantial potential to enhance healthcare teamwork by supporting clinical decisions, streamlining administrative workflows, and improving patient communication, however, ethical, legal, and accountability concerns remain.
Ilse Super, Olya Rezaeian, Onur Asan· International Journal of Med...· 0 citations
A dual-view approach that connects clinical practice with computational methods is presented, establishing a five-level competency scheme following Miller’s Pyramid and linking deductive, inductive, and abductive reasoning patterns to common medical goals and tasks.
GPT-5 reliably adapts linguistic style to clinical personas but produces limited specialty-specific output diversity, supporting its role as a decision-support adjunct rather than an autonomous specialist simulator.
Derna Stifini, A. Della Penna, André L. Mihaljević et al.· npj Digital Medicine· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.