Personalised Drug Recommender Systems based on Electronic Health Records: Reproducibility and External Validity

Abstract

Recommender systems have achieved success in delivering personalised experiences across large-scale consumer platforms, particularly in domains such as entertainment and social media. Recently, there has been increasing interest in applying recommender system methodologies to domains with positive social impact, including healthcare. Although machine learning has demonstrated strong performance across a variety of clinical tasks, comparatively less attention has been given to recommender-style decision support for personalised care. The introduction and increasing adoption of Electronic Health Records (EHRs) has enabled the development of models that leverage patient histories to generate personalised clinical recommendations. In particular, EHR-based drug recommendation aims to predict safe and effective medication combinations using diagnoses, procedures and past visit information. In this work, we examine the current state of research in this area through an analysis and reproducibility study of representative models from the literature. We show that the field has grown rapidly in recent years and increasingly relies on complex deep learning architectures. However, our study reveals a stark gap between code availability and executability: despite nearly 88% of papers providing public code, half of the executable pipelines failed entirely. We further demonstrate that only 2 out of 41 surveyed papers are fully reproducible, and we identify major questions regarding the real-world applicability of the proposed models. By highlighting these issues, we aim to provide a clearer understanding of the field and provide a foundation for more robust and clinically meaningful research in healthcare recommender systems.