Multi-Label Classification of Adverse Drug Reactions Using a Hybrid Approach Integrating Support Vector Machine, Random Forest and Extreme Gradient Boosting

Abstract

Background/Objectives: Predicting adverse drug reactions (ADRs) is essential for drug safety, but challenging due to the rarity and diversity of certain clinical manifestations. This study aims to enhance multi-label ADR detection by introducing and evaluating biologically inspired strategies, specifically Balanced Thresholds (BT) and Weighted Predictions (WP), integrated into a hybrid ensemble model. Methods: We proposed and developed a hybrid architecture combining Random Forest (RF), Extreme Gradient Boosting (XGB), and Support Vector Machine (SVM). To handle rare ADRs without data leakage, balanced thresholds and weighting parameters were calculated exclusively on training folds within a repeated 5 × 5-fold cross-validation framework. Statistical validation was performed using Friedman and Nemenyi tests. The main motivation for this hybrid architecture was to emphasize model diversity, complementary learning mechanisms, error reduction, and generalization. Results: The experimental results demonstrate that the adjusted strategies significantly increase recall compared to baseline models, particularly for rare ADRs. Specifically, the Weighted Predictions strategy achieved near-perfect recall (up to 0.99) for low-prevalence disorders like RSBD and ELD. While the baseline model maintained higher overall precision, the adjustments narrowed the performance gap between frequent and rare reactions, doubling the recall for the most challenging categories. Friedman tests confirmed significant differences in model sensitivity. Conclusions: Integrating biological knowledge into hybrid architectures provides a framework that allows practitioners to prioritize sensitivity in high-risk scenarios and improve the detection of clinically relevant, rare ADRs.