Development of a Framework for Deidentified Japanese Electronic Health Record Narratives Using BERT: Balancing Privacy Protection and Reproducible Entity Extraction in Real-World Data.

Background

Pharmacoepidemiologic studies using real-world data often lack clinical context, much of which is embedded in free-text electronic health records (EHRs). However, sharing EHR narratives is restricted by privacy requirements, and conventional deidentification approaches may reduce analytic utility and limit auditability.

Objective

To develop and evaluate a framework that (1) extracts structured cancer-related entities from Japanese EHR free text using BERT (Bidirectional Encoder Representations from Transformers)-based natural language processing (NLP) and (2) generates deidentified analytic text that preserves contextual information to support post-hoc auditing under privacy constraints.

Methods

We conducted a retrospective observational study using the DATuM IDEA database, which integrates unstructured EHR narratives with structured records (claims, prescriptions, and procedure/surgery records) from two hospitals in Japan from January 1, 2019, through June 30, 2025. Deterministic linkage was performed within ICI (Integrated Clinical Care Informatics, Inc.) prior to deidentification. The source text population comprised all linked progress notes from eligible oncology patients. Downstream recoverability analyses were conducted within a governance-constrained analyzable text cohort derived after deidentification and preprocessing. We quantified deidentification using masking rates and residual visible character density and evaluated downstream recoverability of TNM/stage-related mentions, internal logical consistency (M1 vs Stage IV), and proxy-based concordance using structured-data treatment proxies (ATC code L for systemic anticancer therapy and procedure-name keyword searches for cancer-directed interventions). A stratified manual audit of 400 deidentified narratives was conducted primarily to inspect for apparent residual direct identifiers, with retention of intended TNM and stage outputs reviewed as a secondary sanity check.

Results

Among 51,876 patients with recorded diagnoses, 10,214 had neoplasms (ICD-10 C00-D48), and linked progress notes were available for 4,383 patients (82,863 records). The downstream analyzable text cohort (Layer C), used for downstream recoverability and proxy-based concordance analyses, comprised 3,689 patients and 38,841 documents. Of these, 1,339 patients (36.3%) and 11,914 documents (30.7%) had at least one recoverable TNM or stage mention (Layer D). Across all Layer C documents, marginal document-level recoverability was 24.2% for T, N, and M elements and 23.6% for stage. Conditional on Layer D, the corresponding recoverability rates were 0.788 for T, 0.787 for N, 0.788 for M, and 0.768 for stage. Median masking rates ranged from 9.9% to 15.0% across major cancer categories. M1 and Stage IV mention indicators showed an overall agreement of 94.7%, a positive percent agreement (PPA) of 54.8%, and a negative percent agreement (NPA) of approximately 99.0%. Proxy-based concordance analysis showed a positive predictive value (PPV) of 0.700 for Stage IV using the systemic therapy proxy and a PPV of 0.031 for early-stage classification using the procedure proxy. Manual audit demonstrated retention of intended TNM/stage-related outputs after normalization and identified no apparent residual direct identifiers in the audited deidentified narratives.

Conclusions

This framework enables the generation of deidentified Japanese EHR narratives that preserve contextual structure for auditing while supporting structured entity extraction, thereby addressing the trade-off between privacy protection and reproducibility in real-world data research. CLINICALTRIAL