HEART-Synth: A Synthetic Resident-Patient Dialogue Dataset for Medical Error Disclosure Assessment

Abstract

Effective disclosure of medical error requires factual clarity, accountability, empathy, and a prevention-oriented plan. Automated assessment is challenging as clinically sensitive human-rated disclosure interactions are scarce and consented. This study presents HEART-Synth, a 400-transcript resident-patient disclosure synthetic dataset for scoring development-stage rubrics. Each conversation is presented as transcript with multi-turn dialogue and ordinal scores ranging from 0 to 3 for the HEART domains used in our medical error disclosure curriculum. The dataset was produced with independent perdomain performance sampling, separated dialogue generation and blind scoring, conservative second-pass review, duplicate rejection, and audit-only metadata excluded from model inputs. It is evaluated with audits for scenario-score coupling, lexical shortcuts, adversarial masking, linguistic diversity, and score coverage. HEART-Synth lowers the mean normalized mutual information between scenario identification and score from 0.825 to 0.068 when compared to an initial simple-generator baseline. Linguistic diversity (distinct $-2=0.265;$ self-BLEU $=0.146)$ is preserved, all score levels in every domain are covered, and the best TF-IDF lexical baseline macro-F1 is reduced from 0.886 to 0.560. These findings support the technical quality and development-stage use of HEART-Synth, though future applications should integrate human score validation to ensure clinical validity.