CriticExpert: A Lightweight Task-Specific Verification-and-Correction Framework for LLM-Generated Closed-Set Specialized Healthcare Text Coding

Abstract

Large language models (LLMs) have shown promise for healthcare text coding through in-context learning, yet their generated codes may still suffer from hallucination, inconsistency, and overconfidence. Although LLM-as-a-judge methods provide a flexible way to evaluate generated outputs, they are often computationally expensive, prompt-sensitive, and insufficiently adapted to specialized closed-set coding guidelines. This limitation is particularly important in healthcare text coding, where candidate outputs must conform to predefined label spaces and subtle coding rules. To address this problem, we propose CriticExpert, a lightweight task-specific verification-and-correction framework for LLM-generated closed-set healthcare text coding. CriticExpert uses a pretrained small language model (SLM) as the backbone and introduces two supervised task heads: a correctness-verification head that determines whether an LLM-generated candidate label is correct, and a correction head that predicts the corrected label from a fixed task-specific label set when an error is detected. To improve deployment efficiency, we further investigate knowledge distillation to compress the critic while preserving verification performance. Experiments on specialized healthcare coding datasets show that CriticExpert improves the reliability of LLM-generated candidate codes while substantially reducing inference cost compared with large-model judges. The results suggest that lightweight task-specific critics can serve as practical post-hoc verification modules for specialized closed-set coding workflows. The implementation is publicly available at https://github.com/Toby28/CriticExpert