Adaptive Hybrid Feature Selection and Explainable Stacking Ensemble Learning for Early Chronic Kidney Disease Prediction

Abstract

One type of progressive disease in the world is chronic kidney disease (CKD), where people are commonly diagnosed only in late stages with many severe complications, which further raises the health cost. To minimize the burden of CKD, a novel approach with an intelligent decision support system will help the patients be identified in early stages and prompt the intervention. But most machine learning algorithms have the problem of having duplicate clinical attributes, sparse data, and limited interpretation. This paper introduces an explainable machine learning framework to support timely detection of chronic kidney disease using medical feature engineering, adaptive hybrid feature selection (AHFS), and a stacking ensemble classifier. Missing values in the data are imputed using KNN. After that, clinical attributes were preprocessed and encoded. Medical feature engineering was carried out to construct new clinical features from the dataset. The proposed AHFS algorithm performs the selection of best attributes using an adaptive voting strategy by combining mutual information, ANOVA, random forest and XGBoost feature importance, and recursive feature elimination. Finally, these selected features are trained using a stacking ensemble. Moreover, SHAP is applied to explain the model predictions and contribute features for early CKD prediction. The performance of the proposed system on the UCI Chronic Kidney Disease dataset confirms its excellent performance with minimum redundancy and high interpretability of the model. A comparative study of our framework with conventional machine learning techniques proves the superiority of our proposed framework for early CKD diagnosis with excellent clinical decision support.