Integrating knowledge graphs and retrieval-augmented generation to identify clinical pharmacology gaps in infectious disease trial protocols

Abstract

Clinical trial protocols contain critical clinical pharmacology information that is often scattered across different sections of these lengthy documents. The purpose was to evaluate whether a retrieval-augmented generation (RAG) protocol review integrated with knowledge graphs (KGs) could be used to obtain clinical pharmacology evidence and identify regulatory gaps. Clinical trial protocols were obtained from the Gates Foundation’s publicly available corpus of clinical trials in low- and middle-income countries. The RAG protocol review was used to assess the protocols against Food & Drug Administration (FDA) guidance documents for hepatic impairment, exposure-response, population PK (PopPK), and physiologically based pharmacokinetic modeling (PBPK). Disease-specific KG analyses were conducted for tuberculosis (TB), human immunodeficiency virus (HIV), malaria, and neglected tropical diseases (NTD). KG models connecting studies to drugs, diseases, pharmacologic classes, organ-function exclusions, biomarkers, and pharmacokinetic (PK) parameter features were obtained. Across 42 unique trials, the KG contained 223 nodes and 376 edges. TB protocols produced the largest disease-specific graph, whereas HIV had the highest frequency of PK studies. Gap analysis showed that hepatic impairment exclusions in TB were more frequent than PK studies. In the multi-guidance evaluation, faithfulness was high, answer relevancy was lower, and the clinical pharmacology metric showed guidance-dependent differences. The integrated KG-RAG approach can help highlight potential gaps in clinical pharmacology reporting in infectious disease clinical trial protocols.