Skip to content
Preprint

High-Stakes Decisions with Language Models: Insights from Emergency Triage

Aug 2026 · 1 citation · 24 references
Computer Science

TL;DR

It is suggested that language models for high-stakes applications should be understood and evaluated as probabilistic decision systems whose recommendations depend jointly on predictive performance and explicit utilities.

Abstract

High-stakes decisions under uncertainty, such as medical emergency triage, require more than accurate predictions. They depend on estimating the likelihood of alternative outcomes while explicitly weighing the consequences of different actions, principles that have long formed the foundation of medical diagnosis and decision making. Yet language models are increasingly used for high-stakes clinical recommendations without explicit specification of the utilities governing these decisions. Here we show that emergency triage with language models can be understood within a probabilistic decision framework, providing a case study of a broader decision-analytic paradigm for steering, evaluating, and deploying language models in high-stakes settings. Using clinical vignettes from a structured evaluation of a consumer triage system, we analyze recommendations for treatment under alternative utility functions that specify the relative costs of missed emergencies and unnecessary escalation. We find that capable language models adjust recommendations in response to stated utilities, revealing that the same underlying predictions can support markedly different decision policies. These findings show that effective deployment depends not only on improving predictions but also on making decision objectives explicit. More broadly, they suggest that language models for high-stakes applications should be understood and evaluated as probabilistic decision systems whose recommendations depend jointly on predictive performance and explicit utilities.

View source

Similar papers

Preprint Aug 2026

Rare Diseases, Common Dilemmas: LLMs Prioritize Equal Resource Distribution over Patient Benefit in Decision-Making

This work presents a benchmark of 208 clinically grounded rare disease vignettes and identifies a strong authority-framing effect: models favor justice in committee-based contexts and shift toward beneficence and autonomy only when final decisions are framed as being made by clinicians or patients respectively.

Min-Da Zhao, Xuheng Han, R. Goel et al. · 0 citations
Review Open access Sep 2026

Treating the incumbent patient in triage: An analysis of "first come, first served" as a procedural rule.

Allocating scarce intensive care resources requires decisions about both which patients should receive priority and which rules should guide those choices. We investigate how the procedural allocation rule "first come, first served" (FCFS) shapes triage decisions. Using experimental ICU bed allocation scenarios, we exa...

Iván Aranzales, H. Chan, David Stadelmann et al. · 0 citations
Open access Aug 2026

Prescribing Decisions Using AI-Based Prescriptive Analytics: Procedure, Challenge, and Validation Pathways

Prescriptive analytics uses predictions to recommend decisions, but its superiority over the current approach is commonly evaluated on historical data in which only the cost of the implemented decision is observed. When a prescribed decision differs from the historical decision, its actual cost is a counterfactual outc...

Yunxiang Shu, Ran Yan, Feng Tian et al. · 0 citations
Open access Aug 2026

Value-of-Information Question Selection for Symptom Triage with Missing Patient Evidence

The advent of automated digital health platforms has revolutionized the initial stages of patient care, primarily through the deployment of symptom triage systems. However, a pervasive challenge in these automated environments is the prevalence of missing patient evidence, as individuals often fail to report complete o...

Catherine Turner · 0 citations
#artificial intelligence Preprint Oct 2026

Counterfactual Auditing of Bias in Open-Source Large Language Models for Clinical Triage

Emergency department (ED) triage is a high-stakes prioritization task in which demographic, socioeconomic, and system-context information may improperly influence acuity assignment. Although open-source large language models (LLMs) are increasingly considered for local and privacy-preserving clinical decision support,...

Manar Aljohani, Brandon Ho, Kenneth W. McKinley et al. · 0 citations
Preprint Aug 2026

Proxy reliance in large language model decisions is uncalibrated to predictive evidence

Large language models are entering decisions in triage and lending, where task-relevant inference must be distinguished from impermissible proxy use, and causal proxy effects in four LLMs on a clinical-ranking task with known ground truth are measured.

Zeng-Qing Wu, Chuan Xiao · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.