Causality-Based Conformal Imputation Correction with Non-Random Missing Labels
TL;DR
This work proposes a model-agnostic framework to assess the accuracy of imputed labels and to correct imputations with large bias based on conformal prediction, and leverage conformal prediction to construct a valid prediction set for units with unobserved labels, and revise imputations that fall outside this set.
Abstract
Collected data with non-random missing labels poses a widely recognized challenge for unbiased learning. For example, in recommender systems, users are free to choose whether or not to rate an item. To achieve unbiased learning under MNAR data, a variety of methods have been proposed, such as reweighting and imputation. Among them, doubly robust (DR) based methods are widely adopted due to their appealing theoretical guarantees. However, these guarantees rely on strong assumptions that either the propensity or the imputation is accurate for all units (such as user-item pairs), which is very hard to achieve in real-world scenarios. Previous studies show that a small error in imputation can lead to a large bias in DR-based methods. Furthermore, for units with missing labels, we lack an effective method to evaluate the imputation quality. In this work, we propose a model-agnostic framework to assess the accuracy of imputed labels and to correct imputations with large bias based on conformal prediction. Specifically, we leverage conformal prediction to construct a valid prediction set for units with unobserved labels, and revise imputations that fall outside this set. Extensive experiments are conducted on three real-world datasets and one semi-synthetic dataset to show the effectiveness of our proposed method. Our code is available at https://github.com/lixiang-222/conformal-prediction-for-MNAR.