An explainable AI framework for multimodal student performance prediction and institutional decision support at PSAU

Abstract

Improving student success and supporting evidence-based decision-making in higher education depend on early detection of academically at-risk students. An Explainable Artificial Intelligence (XAI) framework is proposed in this paper for multimodal student performance prediction to support institutional decision-making at Prince Sattam bin Abdulaziz University (PSAU). The framework was developed and evaluated using a publicly available educational dataset collected from a higher education institution, integrating data from the Student Information System (SIS), Moodle Learning Management System (LMS), and the eDify mobile learning application. The proposed framework combines comprehensive data preprocessing, Mutual Information-based feature selection, Optuna-based hyperparameter optimization, and an optimized CatBoost classifier. Using repeated stratified 5-fold cross-validation, the model's performance was assessed and compared with statistical learning methods, conventional machine learning algorithms, homogeneous ensemble methods, and heterogeneous ensemble approaches. The proposed model demonstrated superior predictive efficacy, attaining an Accuracy of 92.0%, Precision of 92.7%, Recall of 97.9%, F1-score of 95.2%, ROC-AUC of 0.913, PR-AUC of 0.974, Balanced Accuracy of 0.824, and Matthews Correlation Coefficient of 0.726, significantly exceeding all rival models according to the Wilcoxon signed-rank test (p < 0.001). Mutual Information feature selection further improved predictive performance while reducing model complexity. Explainability was achieved through CatBoost feature importance and SHAP analyses, which consistently identified assessment outcomes and online learning engagement variables as the most influential predictors. The proposed framework provides a transferable blueprint for implementing explainable early-warning systems at Prince Sattam bin Abdulaziz University and other higher education institutions to support timely interventions, personalized learning support, and evidence-based institutional decision-making.