A counterfactual harm-aware policy learning framework that effectively balances expected reward with individual-level harm is proposed and an additive latent variable model under which the identifiability of THR and THQ and the corresponding estimators are established and used to learn policies under explicit harm constraints are introduced.
Abstract
Offline policy learning aims to optimize individualized decisions using historical data. However, conventional methods primarily focus on maximizing expected rewards while neglecting individual-level counterfactual harm—cases where the assigned treatment leads to worse outcomes than the control. This can result in overly aggressive policies that harm some individuals. In this article, we propose a counterfactual harm-aware policy learning framework that effectively balances expected reward with individual-level harm. To quantify harm, we introduce two measures: treatment harm rate (THR), capturing the proportion of individuals harmed, and treatment harm quantity (THQ), capturing the severity of harm, particularly for continuous outcomes. A key challenge is that both THR and THQ depend on the joint distribution of potential outcomes, which is generally unidentifiable, even in randomized controlled trials. To address this, we introduce an additive latent variable model under which we establish the identifiability of THR and THQ and develop the corresponding estimators. These estimators are then used to learn policies under explicit harm constraints. Extensive experiments demonstrate the effectiveness of the proposed approach.
This paper studies risk-averse treatment allocation when individuals self-select into treatment based on unobserved characteristics. We develop a framework that combines the marginal treatment effect approach to endogenous selection with a general class of coherent risk measures that capture distributional preferences...
Policy evaluation and learning aim to assess and optimize treatment assignment rules based on individual characteristics. A fundamental challenge arises when outcomes are truncated by death, rendering conventional policy utilities undefined. To address this challenge, we study policy evaluation and learning under trunc...
Conformal policy learning (CPL) is proposed, a policy learning procedure with a new distribution-free safety guarantee that controls the probability of assigning treatment to an individual who would be harmed relative to control and achieves asymptotically optimal welfare subject to the safety constraint.
Off-policy evaluation (OPE) for contextual bandit policies becomes challenging when action-level importance weighting incurs excessive variance. Doubly robust (DR) estimation remains unbiased under common support but retains these high-variance action-level weights. A prior estimator, Off-policy evaluation with Conjunc...
Nicolò Felicioni, Michael Benigni, Maurizio Ferrari Dacrema et al.· 0 citations
Policy learning methods based on conditional average treatment effects can obscure subpopulation heterogeneity when applied to ordinal outcomes. We develop a policy learning framework for ordinal outcomes with heterogeneous utilities for individuals who strictly benefit from treatment and those who do not. Since the pr...
This paper proposes a method for policy learning called policy-vote bagging, which learns treatment decisions on many subsamples then averages their votes into treatment probabilities, which preserves expected welfare and improves expected utility for a risk-averse researcher.