AMIUgraph: analysis and modeling of interactions for utility-driven benchmarking of graph-based models in healthcare.
TL;DR
AMIUGraph provides a clinically grounded and utility-driven benchmarking framework that jointly evaluates KGE and GNN models across multiple healthcare graph types and learning settings, and offers practical guidance for selecting graph-based models in medical decision-support systems.
Abstract
Background
Graph-based machine learning approaches, including Knowledge Graph Embedding (KGE) methods and Graph Neural Networks (GNNs), have emerged as powerful tools for modeling complex biomedical data. However, a systematic and clinically grounded comparison of these approaches across heterogeneous healthcare graphs, accounting for both predictive performance and real-world deployment constraints, is still lacking.
Methods
We introduce AMIUGraph, a comprehensive benchmarking framework for healthcare link prediction that integrates real-world clinical data with external biomedical knowledge bases. AMIUGraph evaluates eight state-of-the-art models, of which four are knowledge graph embedding (KGE) methods DistMult, CP, ComplEx, and ConvE and four are graph neural network (GNN) architectures GCN, GraphSAGE, GAT, and GIN. The models are evaluated across three heterogeneous bipartite graphs representing Patients-Diseases, Diseases-Drugs, and Drugs-Targets interactions. Models are assessed under both transductive and inductive learning settings using accuracy, AUC, precision, recall, F1-score, and training time as evaluation metrics.
Results
Experimental results show that model performance is strongly influenced by graph structure and sparsity. GNNs consistently achieve superior predictive performance on sparse interaction graphs, particularly for Diseases-Drugs and Drugs-Targets prediction tasks. In contrast, KGE models demonstrate competitive accuracy with substantially lower computational costs in inductive clinical scenarios involving unseen patients. These trends are especially relevant in clinically realistic settings characterized by multimorbidity, such as gastrointestinal and liver diseases, where frequent patient updates and complex therapeutic interactions are common.
Conclusion
AMIUGraph provides a clinically grounded and utility-driven benchmarking framework that jointly evaluates KGE and GNN models across multiple healthcare graph types and learning settings. The findings offer practical guidance for selecting graph-based models in medical decision-support systems, including applications in gastrointestinal healthcare, while promoting transparency and reproducibility through the public release of all datasets, protocols, and code.