Skip to content

Decomposing Error and Style in Automated Clinical Coding

Sep 2026 · 0 citations · 18 references
Computer Science

TL;DR

This work model the systematic component as coding style $\psi$, a coder- or site-specific policy over what to code and how much to document, and recast coding as $p(\mathrm{code}\mid\mathrm{note},\psi)$, estimating $\psi$ with a 10-dimension rubric.

Abstract

In automated clinical coding, where the label space spans tens of thousands of diagnosis and procedure codes, models are currently evaluated against a single gold annotation, treating any deviation as error. But we find when two teams code the same 110 ACI-Bench encounters, they agree on only 73% of codes (Jaccard similarity) for the same note; even after an independent clinical audit removes erroneous codes, agreement rises only to 77%. Is that gap error or something systematic? We model the systematic component as coding style $\psi$, a coder- or site-specific policy over what to code and how much to document, and recast coding as $p(\mathrm{code}\mid\mathrm{note},\psi)$, estimating $\psi$ with a 10-dimension rubric. If style were noise, conditioning on it would do nothing. Instead, across five datasets a model conditioned with a data-matching style raises ICD F1 by up to 26 points and an extreme mismatched one lowers it by up to 21. Four prompt based coding methods spanning 39-49 F1 converge to 52-56 once style is supplied (All p<0.05). Much of what single-gold evaluation charges to model error is recoverable, unmodeled style.

View source

Similar papers

#natural language process... Preprint Aug 2026

KREL: Automatic Medical Coding via Knowledge-Guided Reasoning over Clinical Evidence with LLMs

This work proposes Knowledge-Guided Reasoning over Clinical Evidence with LLMs (KREL), a framework that leverages LLMs for clinical text understanding and reasoning while integrating external ICD coding guidelines as structured knowledge, and enables tight coupling between domain knowledge and LLM reasoning.

Xubin Chen, Yi-Peng Zhou, Wenxin Sun et al. · 0 citations
Preprint Aug 2026

Toward Better Assessment of LLMs'Performance in Clinical Error Detection

While models consistently locate error-relevant content, they fail to produce the corresponding correct verdict on the clean counterpart and it is shown that F1 and pairwise accuracy are driven in opposite directions by the same underlying bias, so that ranking models by F1 may systematically promote the weakest discri...

Yifan Zhang, Rahmatollah Beheshti · 0 citations
Aug 2026

Large Language Models vs. NASS Common Coding Scenarios: An Analysis of Accuracy and Financial Implications using 2026 CPT AMA Codebook Guidelines.

This study evaluates baseline performance under isolated zero-shot conditions; the tendency of Claude and ChatGPT to create unbundled billable codes presents a severe audit risk, while Gemini demonstrated a lower mean financial variance.

Christopher J. Fang, J. L. Hii, Matthew Michelberger et al. · 0 citations

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.