Learning human aligned explanations through supervised aggregation of XAI methods for Arabic sentiment analysis

Abstract

Explainable Artificial Intelligence (XAI) aims to make complex AI models more transparent by revealing how their predictions are made. However, XAI methods such as LIME and SHAP often yield inconsistent results, as each captures different aspects of model behavior. In this work, we propose a meta-model framework that learns to combine XAI outputs under human supervision for Arabic sentiment analysis. Using human-annotated datasets and a transformer model as the base black-box model, we train three supervised meta-models (Logistic Regression, XGBoost and CatBoost) on token-level features derived from multiple XAI scores, transformer attention weights, embeddings, and other linguistic features. Results show that the supervised meta-model achieves higher plausibility (human alignment) than the individual XAI methods. The CatBoost meta-model reaches an F1 score of 0.84, compared with 0.64−0.70 for LIME, SHAP, IG, DeepLIFT, and Ensemble XAI. However these gains come with a trade-off in faithfulness, where the meta-model scores are lower than our previously proposed Ensemble XAI on our combined faithfulness metric (0.61 for XGBoost meta-model vs. 0.76 for Ensemble XAI). To the best of our knowledge, this study provides the first investigation of human-supervised XAI aggregation in Arabic, offering new insights into balancing faithfulness and plausibility in explainable NLP. The code is publicly available at: https://github.com/ychafiqui/xai-meta-model