Aug 2026· The spine journal· 0 citations· 13 references
Medicine
TL;DR
This study evaluates baseline performance under isolated zero-shot conditions; the tendency of Claude and ChatGPT to create unbundled billable codes presents a severe audit risk, while Gemini demonstrated a lower mean financial variance.
Abstract
Background
Context
Accurate Current Procedural Terminology (CPT) coding is essential for compliant revenue cycle management in spine surgery. However, increasing documentation burdens and the complexity of bundling rules often lead to significant revenue leakage or audit risk.
Purpose
To evaluate the baseline efficacy of three state-of-the-art Large Language Models (LLMs) in generating accurate CPT codes compared to the "Gold Standard" North American Spine Society (NASS) Common Coding Scenarios, and to quantify the downstream financial impact.
STUDY DESIGN/
Setting
Comparative artificial intelligence performance analysis.
PATIENT SAMPLE
Twenty standardized clinical vignettes selected from the NASS Common Coding Scenarios guide, representing a broad spectrum of spine pathology including cervical decompression/fusion, lumbar decompression, lumbar fusion, and complex deformity.
OUTCOME MEASURES
Exact match accuracy, error of omission (under-coding), error of commission (over-coding/unbundling), and net financial variance utilizing 2026 Medicare Physician Fee Schedule (MPFS) Work RVU (wRVU) values mapped to a standardized commercial conversion factor of $60/wRVU.
Methods
Each vignette was input into three LLMs: GPT-4o (OpenAI), Gemini 1.5 Pro (Google), and Claude 4.6 Sonnet (Anthropic). Under isolated zero-shot conditions, the models were prompted to function as certified professional coders and generate the appropriate CPT codes and modifiers. Outputs were scored against the official NASS answer key, and subsequent financial variances were calculated.
Results
Gemini 1.5 Pro achieved an exact match rate of 65%, followed by ChatGPT-4o (55%) and Claude 4.6 Sonnet (40%), though these differences in overall performance were not statistically significant (p = 0.28). However, performance varied substantially by procedure type; notably, both ChatGPT and Claude achieved a 0% exact match rate for Lumbar Fusion constructs, frequently failing to correctly bundle interbody and posterior instrumentation codes. Financially, Gemini 1.5 Pro was the most stable, resulting in a negligible mean net revenue variance of -$2.16 per case. In contrast, ChatGPT and Claude demonstrated a strong tendency toward over-coding and high financial volatility, resulting in mean net revenue overcharges of +$75.60 and +$104.07 per case, respectively.
Conclusions
While no statistically significant difference in overall accuracy was detected between models, qualitative deficits remain severe in handling complex fusion hierarchies. This study evaluates baseline performance under isolated zero-shot conditions; the tendency of Claude and ChatGPT to create unbundled billable codes presents a severe audit risk, while Gemini demonstrated a lower mean financial variance. Currently, LLMs should function only as adjunctive tools requiring strict human supervision.
Background: Large language models (LLMs) like ChatGPT show their potential use in healthcare. However, limited research has been conducted on how effective these models are in hand surgery. Therefore, the purpose of this study is to compare ChatGPT-4, ChatGPT-4o and ChatGPT-o1 across patient education, billing and diag...
S. Daulat, Romir P. Parmar, Tyler T. Brady et al.· The Journal of Hand Surgery...· 0 citations
PURPOSE
Most clinical evaluations of large language models assess factual recall rather than the multi-step reasoning behind operative plans. Reasoning-tuned models, post-trained to generate explicit intermediate reasoning before answering, may better approximate surgical decision-making. We compared two such models fr...
C. Lam, Conor T. Boylan, Adit Ravishankar et al.· Spine Deformity· 0 citations
BACKGROUND
National Surgical Quality Improvement Program (NSQIP) data collection depends on labor-intensive manual chart abstraction, limiting efficiency, increasing cost, and necessitating patient sampling. This study evaluated whether a large language model (LLM) could accurately abstract unstructured NSQIP breast re...
Tahera Alnaseri, Yasmine Ibrahim, Ansgar Grunseid et al.· Journal of the American Coll...· 0 citations
This work model the systematic component as coding style $\psi$, a coder- or site-specific policy over what to code and how much to document, and recast coding as $p(\mathrm{code}\mid\mathrm{note},\psi)$, estimating $\psi$ with a 10-dimension rubric.
Han-Chin Shing, Jack Moriarty, Ryan Ware et al.· 0 citations
Current-generation LLMs show high and comparable performance in addressing complex KT-related clinical questions and their consistent accuracy, quality, and readability support their potential role as adjunctive clinical decision-support tools, provided appropriate physician oversight.
M. Polat, Mecit Çelik, Kazım Ceviz et al.· World journal of urology· 0 citations
LLM-based chatbots can provide acceptable surgical guidance for straightforward primary surgical cases, but their utility is limited in high-risk or complex clinical settings, and their rationale quality was significantly lower in complex scenarios than in primary ones.
Müge Toprak, Büşra Yılmaz Tuğan, N. Yüksel· Seminars in Ophthalmology· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.