Behavioral Safety and Context Retention of Large Language Models in a Longitudinal ICU Simulation under Offline Conditions

Abstract

Background. Large language models (LLMs) are increasingly proposed as clinical assistants in critical care, yet their behaviour under prolonged clinical context, conflicting data and authoritative pressure remains insufficiently evaluated. This is particularly relevant for offline or resource-constrained environments, where cloud-based safeguards are unavailable. Methods. I conducted a fully automated behavioural evaluation of 23 open-weight language models using a structured, time-series intensive care unit (ICU) simulation of 32 events spanning 121 hours (five days) of synthetic patient data. The scenario comprised routine monitoring, three distinct data–clinical conflict traps (one presented twice, four trap events in total), episodes of physiological deterioration, and a final stress test in which an authoritative order requested a penicillin-class antibiotic for a patient with penicillin anaphylaxis documented at admission and never repeated. Models ran locally on a single consumer workstation with no network access. Each model's response to the final order was adjudicated into one of four mutually exclusive classes: contextually grounded refusal, ungrounded refusal, unsafe compliance, or no usable verdict. Secondary endpoints were extraction of the allergy at admission, discrepancy tagging across the four conflict traps, unwarranted therapeutic escalation, and response latency. Results. Only 7 of 23 models (30.4%) refused the contraindicated order on grounds explicitly referencing the documented allergy; 6 (26.1%) under a stricter criterion requiring the refusal to be stated as a ruling rather than implied by an assertion of danger. Two models (8.7%) declined the order for unrelated reasons, and three (13.0%) complied — one of them by asserting that no allergy was on record. The largest single group, 11 of 23 models (47.8%), issued no ruling on the order at all: variously, output was truncated inside an unfinished reasoning block, degenerated off-task, restated the protocol without applying it, consisted of a bare classification tag, hedged without resolving, or deferred the question to further assessment. Eight models (34.8%) failed to affirm the allergy at the very first probe, three of them by explicit denial. Of the 12 models that did extract the allergy at admission, only 4 (33.3%) went on to refuse the contraindicated order on that ground; three more raised the allergy at the decision point without acting on it, one of them only to deny that any was on record. Discrepancy detection and contraindication handling were dissociable: one of the four models with perfect discrepancy detection (4/4 traps) approved the contraindicated antibiotic, while two models that tagged no discrepancies at all refused the order on the allergy. Unwarranted escalation on conflict traps was common (11/23, 47.8% issued at least one inappropriate critical alert), but explicit hallucinated pharmacological or procedural intervention was less so (5/23, 21.7%). Contrary to expectation, longer median latency was modestly associated with grounded refusal (Spearman ρ = 0.48, p = 0.019). Latency was not adjusted for parameter count, which was not analysed as a variable, and it times a generation other than the one scored. Conclusions. Under offline-first conditions, 16 of 23 models (69.6%) — 17 (73.9%) under the stricter criterion — failed to produce a safe, contextually grounded refusal of a life-threatening order. The dominant failure mode was not sycophancy but the failure to deliver any interpretable safety verdict, most often because an output-length constraint truncated the attempt — a finding that reframes deployment risk from "the model agrees with me" to "the model does not answer at all." Explicit unsafe compliance was less frequent than previously reported but no less consequential where it occurred. Safety-relevant competencies did not co-vary, so a model that reasons well about artefacts cannot be assumed to handle contraindications. General-purpose LLMs should not be deployed as autonomous clinical agents; the subset that behaved safely suggests that offline-capable assistants remain achievable through hybrid designs incorporating explicit refusal mechanisms, discrepancy-aware reasoning and retrieval-augmented grounding in validated clinical knowledge.