This work presents a benchmark of 208 clinically grounded rare disease vignettes and identifies a strong authority-framing effect: models favor justice in committee-based contexts and shift toward beneficence and autonomy only when final decisions are framed as being made by clinicians or patients respectively.
Abstract
Clinical decision-making often involves prioritizing ethical values, such as beneficence, non-maleficence, respecting a patient's autonomy, and justice. Recent work has begun to assess how large language models (LLMs) make such subjective, value-laden clinical judgments. However, evaluations of LLM decision-making in rare disease care contexts, where ethical tensions are ubiquitous and where scarce prior information likely impacts LLM behavior, are still lacking. Here, we present a benchmark of 208 clinically grounded rare disease vignettes, each of which presents genuine, high-stakes conflicts. When prompting 11 state-of-the-art LLMs to choose between clinically defensible yet ethically conflicting next steps embedded within these vignettes, we found that all evaluated models consistently prioritized justice over other core bioethical principles. Specifically, models overwhelmingly favor equal resource allocation over need-based considerations, indicating LLMs'limited responsiveness to differences in clinical severity or situational context. We also identify a strong authority-framing effect: models favor justice in committee-based contexts and shift toward beneficence and autonomy only when final decisions are framed as being made by clinicians or patients respectively. Our work suggests that institutional pressures surrounding rare disease resource utilization may be silently reflected in LLM-based decision support systems, with finer ethical considerations disregarded.
This work investigates how large language models reason about responsibility and its consequences, tracing their judgments across successive levels, from the behavior, to the resulting illness, to the denial of care.
Hadi Hosseini, Samarth Khanna, Leona Pierce· Proceedings of the AAAI/ACM...· 0 citations
Clinicians and healthcare organizations must distinguish between protecting patients from unwanted technological involvement and requiring institutions to provide additional resources or alternative workflows to ensure that patient preferences are addressed consistently without compromising safety, fairness, privacy, o...
M. Temsah, R. Muaygil, Kamran Sattar et al.· Pakistan Journal of Medical...· 0 citations
This article explores how rare diseases reveal the moral and epistemic foundations of healthcare systems. Based on narrative interviews with 11 Chilean patients and 10 professionals, this study shows how the absence of standardised protocols-such as clinical guidelines, treatment pathways and referral standards-exposes...
Nicolás Schöngut-Grollmus, Dolores Steverlynck, Fernanda Manríquez-Castillo et al.· Sociology of Health and Illn...· 0 citations
Substituted judgment is a morally constrained interpretive standard that seeks the decision most faithful to an incapacitated patient's prior values, commitments, and evaluative outlook under acknowledged uncertainty and can help nurses and other clinicians distinguish patient-centered reasoning from adjacent standards...
Healthcare decision‐making under the Mental Capacity Act 2005 (MCA 2005) has long been understood as comprising two distinct, sequential decisions. First, a
clinical
decision about what treatment options are available. Second, a
best interests
decision about which (if any) of these options to proceed with. This h...
J. Parsons, Alex Cisneros· Modern law review· 0 citations
It is suggested that language models for high-stakes applications should be understood and evaluated as probabilistic decision systems whose recommendations depend jointly on predictive performance and explicit utilities.
K. Yamin, Christopher Kelly, Bryan Wilder et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.