HealthGraphBench: Benchmarking Incremental Graph Value in Public Health Regulatory ML

Abstract

HealthGraphBench v0.2.0 is a reproducible benchmark for evaluating whether relational informationand learned graph representations provide predictive value beyond strong entity-local and simplerelational baselines in public health and regulatory machine learning. The frozen v0.1.2 core contains FDA MAUDE product/problem ranking and CMS nursing-homeserious-deficiency prediction. Version 0.2.0 adds CMS Medicare Part D as a bounded cross-domainreplication task. On the frozen MAUDE test, the simple historical-neighbor baseline reaches Recall@10 of 0.192045,compared with 0.172840 for the evaluated GraphSAGE model. On the frozen CMS nursing-home test,adding ownership aggregates changes ROC AUC by +0.001318; the clustered uncertainty intervalincludes zero. On the Part D 2024 held-out test, specialty popularity reaches micro Recall@10 of 0.217860, historyoverlap reaches 0.210373, tabular logistic reaches 0.168188, and graph BPR reaches 0.180789. Part Dpredicts first observed published provider-drug relationships using exact trimmed generic-nameidentity. It does not represent a first prescription or verified prescribing start. Across tasks with an admitted local-only comparator, simple relational heuristics improve theselected task-local point metric. The evaluated learned-relational methods do not establishadditional value beyond the best relational heuristic. Part D has no admitted local-only comparatorand therefore does not contribute to an aggregate claim about relational information versus localinformation. These are development-stage temporal evaluations. The target periods were inspected duringdevelopment and are not untouched prospective confirmation. The results are evidence about thespecified task contracts and source snapshots, not evidence of clinical utility, causal effects, oruniversal graph-model performance. CMS suppresses provider-drug combinations with 10 or fewer Part D claims; absence is thereforenon-observation subject to suppression and left censoring. Raw FDA, CMS, and Part D source datasetsare not redistributed. The archive contains software, task contracts, manifests, derived results,checksums, research documentation, and the immutable Part D model-gate artifacts needed to reproducethe benchmark from official public sources.