Skip to content

The Judgment-Consequence Gap: LLM Moral Reasoning in Healthcare Decisions

Aug 2026 · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · 0 citations
Computer Science

TL;DR

This work investigates how large language models reason about responsibility and its consequences, tracing their judgments across successive levels, from the behavior, to the resulting illness, to the denial of care.

Abstract

As large language models (LLMs) enter high-stakes domains such as healthcare, understanding their moral reasoning becomes essential. Decisions about scarce medical resources often hinge on judgments of responsibility, particularly when patients'own actions contribute to illness. We investigate how LLMs reason about responsibility and its consequences, tracing their judgments across successive levels, from the behavior, to the resulting illness, to the denial of care. We evaluate a wide range of LLMs, spanning different model families and capability levels, on various clinical vignettes adapted from prior studies. Our results identify a judgment-consequence gap: LLMs largely agree with humans that patients bear responsibility for health-harming behaviors, yet overwhelmingly refuse to let that judgment influence how they allocate scarce resources. Specifically, LLMs default to random allocation, whereas humans consistently favor the less-culpable patient. Compared to humans, LLMs also place greater emphasis on access to information, reducing responsibility judgments when health-risk knowledge is unavailable. These findings reveal that LLMs apply a systematically different moral framework than humans when responsibility and resource scarcity intersect, surprisingly often amplifying normative disagreement with humans as reasoning capability increases.

View source

Similar papers

Preprint Aug 2026

Rare Diseases, Common Dilemmas: LLMs Prioritize Equal Resource Distribution over Patient Benefit in Decision-Making

This work presents a benchmark of 208 clinically grounded rare disease vignettes and identifies a strong authority-framing effect: models favor justice in committee-based contexts and shift toward beneficence and autonomy only when final decisions are framed as being made by clinicians or patients respectively.

Min-Da Zhao, Xuheng Han, R. Goel et al. · 0 citations
Open access Sep 2026

Patient Refusal of LLM-Assisted Care: Autonomy in an AI-Enabled Health System

Clinicians and healthcare organizations must distinguish between protecting patients from unwanted technological involvement and requiring institutions to provide additional resources or alternative workflows to ensure that patient preferences are addressed consistently without compromising safety, fairness, privacy, o...

M. Temsah, R. Muaygil, Kamran Sattar et al. · 0 citations
Open access Aug 2026

Rare Diseases as Epistemic Objects: Exploring the Moral Role of Health Systems.

This article explores how rare diseases reveal the moral and epistemic foundations of healthcare systems. Based on narrative interviews with 11 Chilean patients and 10 professionals, this study shows how the absence of standardised protocols-such as clinical guidelines, treatment pathways and referral standards-exposes...

Nicolás Schöngut-Grollmus, Dolores Steverlynck, Fernanda Manríquez-Castillo et al. · 0 citations
Sep 2026

Substituted judgment in health care decision-making: An evolutionary concept analysis.

Substituted judgment is a morally constrained interpretive standard that seeks the decision most faithful to an incapacitated patient's prior values, commitments, and evaluative outlook under acknowledged uncertainty and can help nurses and other clinicians distinguish patient-centered reasoning from adjacent standards...

Claire McGowan · 0 citations
Review Open access Aug 2026

Physicians Retain Moral Responsibility but Endorse Institutional Co-Responsibility in AI-Assisted Decisions: An Exploratory Vignette Study

Physicians treated AI as an instrument rather than a bearer of responsibility, which is unsurprising, and personal responsibility was retained across all decision contexts, while what varied was the admissibility of institutional constraints as excuses and the emphasis placed on the institution's share.

Florian Berghea, Alexandra-Ligia Dincă, D. Ciuc et al. · 1 citation
#artificial intelligence Preprint Sep 2026

Hindsight Bias in Clinical Temporal Reasoning: How Future Data Exposure Affects Large Language Model Judgment

A paired benchmark for measuring outcome-conditioned shifts consistent with hindsight bias in clinical temporal reasoning is introduced and accuracy (Acc), hindsight trap rate (HTR), answer instability rate (AIR), and hindsight bias rate (HBR) are evaluated.

Misaki Matsuura, Sayantan Kumar, Ojas Kadam et al. · 1 citation

Related blog posts

Google DeepMind Blog Nov 18, 2026

Genie 4 is now available

We are rolling out a new Project Genie experience to eligible Google AI Ultra subscribers, opening access to our new Genie 4 world model.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.