Improving Clinical Note De-identification via Post-NER Verification and Candidate Expansion
—De-identification of clinical notes is critical for protecting patient privacy, yet existing approaches often struggle under real-world variation and provide limited support for auditing and error analysis. By examining the outputs of NER-based systems, we observe three recurring failure modes: false positives, false negatives, and fragmented entity spans. The latter can simultaneously introduce both error types under exact-match evaluation. We present a transparent, locally deployable de-identification framework that augments NER-based extraction with two refinement stages: a Verification loop to correct candidate entities and a Candidate Expansion loop to recover missed protected health information (PHI). Beyond improving extraction quality, the system generates structured artifacts for human review, including fine-grained error attribution, audit-ready spreadsheet exports, and an interactive analytical dashboard for cross-configuration comparison, enabling systematic inspection and iterative refinement of de-identification results. We evaluate the framework on the i2b2 2014 benchmark, a MIMIC-IV radiology subset, and a synthetic dataset simulating distribution shift. Results show substantial robustness gains under variation, increasing entity-level F1 from 61.52% to 87.78% and reducing the false-negative rate from 29.75% to 11.59% on the synthetic dataset, while maintaining competitive performance on benchmark data. These findings highlight the value of combining post-NER refinement with transparency-oriented evaluation infrastructure for reliable clinical de-identification.