Aug 2026· Proceedings of the AAAI/ACM Conference on AI Ethics and Society· 0 citations
Computer Science
TL;DR
This work investigates how large language models reason about responsibility and its consequences, tracing their judgments across successive levels, from the behavior, to the resulting illness, to the denial of care.
Abstract
As large language models (LLMs) enter high-stakes domains such as healthcare, understanding their moral reasoning becomes essential. Decisions about scarce medical resources often hinge on judgments of responsibility, particularly when patients'own actions contribute to illness. We investigate how LLMs reason about responsibility and its consequences, tracing their judgments across successive levels, from the behavior, to the resulting illness, to the denial of care. We evaluate a wide range of LLMs, spanning different model families and capability levels, on various clinical vignettes adapted from prior studies. Our results identify a judgment-consequence gap: LLMs largely agree with humans that patients bear responsibility for health-harming behaviors, yet overwhelmingly refuse to let that judgment influence how they allocate scarce resources. Specifically, LLMs default to random allocation, whereas humans consistently favor the less-culpable patient. Compared to humans, LLMs also place greater emphasis on access to information, reducing responsibility judgments when health-risk knowledge is unavailable. These findings reveal that LLMs apply a systematically different moral framework than humans when responsibility and resource scarcity intersect, surprisingly often amplifying normative disagreement with humans as reasoning capability increases.
This work presents a benchmark of 208 clinically grounded rare disease vignettes and identifies a strong authority-framing effect: models favor justice in committee-based contexts and shift toward beneficence and autonomy only when final decisions are framed as being made by clinicians or patients respectively.
Min-Da Zhao, Xuheng Han, R. Goel et al.· 0 citations
Clinicians and healthcare organizations must distinguish between protecting patients from unwanted technological involvement and requiring institutions to provide additional resources or alternative workflows to ensure that patient preferences are addressed consistently without compromising safety, fairness, privacy, o...
M. Temsah, R. Muaygil, Kamran Sattar et al.· Pakistan Journal of Medical...· 0 citations
This article explores how rare diseases reveal the moral and epistemic foundations of healthcare systems. Based on narrative interviews with 11 Chilean patients and 10 professionals, this study shows how the absence of standardised protocols-such as clinical guidelines, treatment pathways and referral standards-exposes...
Nicolás Schöngut-Grollmus, Dolores Steverlynck, Fernanda Manríquez-Castillo et al.· Sociology of Health and Illn...· 0 citations
Substituted judgment is a morally constrained interpretive standard that seeks the decision most faithful to an incapacitated patient's prior values, commitments, and evaluative outlook under acknowledged uncertainty and can help nurses and other clinicians distinguish patient-centered reasoning from adjacent standards...
Physicians treated AI as an instrument rather than a bearer of responsibility, which is unsurprising, and personal responsibility was retained across all decision contexts, while what varied was the admissibility of institutional constraints as excuses and the emphasis placed on the institution's share.
Florian Berghea, Alexandra-Ligia Dincă, D. Ciuc et al.· Applied Sciences· 1 citation
A paired benchmark for measuring outcome-conditioned shifts consistent with hindsight bias in clinical temporal reasoning is introduced and accuracy (Acc), hindsight trap rate (HTR), answer instability rate (AIR), and hindsight bias rate (HBR) are evaluated.
Misaki Matsuura, Sayantan Kumar, Ojas Kadam et al.· 1 citation
An MIT and Dana-Farber Institute-led team used large language models to examine nearly 4,000 studies and hundreds of code repositories, revealing gaps in the information needed to scrutinize and reuse published research.