Skip to content
Open access

Offline Policy Evaluation and Learning with Harm Constraints

Sep 2026 · Entropy · Vol 28, pp. 1058 · 0 citations · 51 references
Medicine

TL;DR

A counterfactual harm-aware policy learning framework that effectively balances expected reward with individual-level harm is proposed and an additive latent variable model under which the identifiability of THR and THQ and the corresponding estimators are established and used to learn policies under explicit harm constraints are introduced.

Abstract

Offline policy learning aims to optimize individualized decisions using historical data. However, conventional methods primarily focus on maximizing expected rewards while neglecting individual-level counterfactual harm—cases where the assigned treatment leads to worse outcomes than the control. This can result in overly aggressive policies that harm some individuals. In this article, we propose a counterfactual harm-aware policy learning framework that effectively balances expected reward with individual-level harm. To quantify harm, we introduce two measures: treatment harm rate (THR), capturing the proportion of individuals harmed, and treatment harm quantity (THQ), capturing the severity of harm, particularly for continuous outcomes. A key challenge is that both THR and THQ depend on the joint distribution of potential outcomes, which is generally unidentifiable, even in randomized controlled trials. To address this, we introduce an additive latent variable model under which we establish the identifiability of THR and THQ and develop the corresponding estimators. These estimators are then used to learn policies under explicit harm constraints. Extensive experiments demonstrate the effectiveness of the proposed approach.

Read PDF

Similar papers

Preprint Sep 2026

Risk-Averse Welfare Maximization via Marginal Treatment Effects

This paper studies risk-averse treatment allocation when individuals self-select into treatment based on unobserved characteristics. We develop a framework that combines the marginal treatment effect approach to endogenous selection with a general class of coherent risk measures that capture distributional preferences...

Jarrod Burgh, Emerson Melo · 0 citations
Preprint Sep 2026

Policy evaluation and learning with partially identified utility under truncation by death

Policy evaluation and learning aim to assess and optimize treatment assignment rules based on individual characteristics. A fundamental challenge arises when outcomes are truncated by death, rendering conventional policy utilities undefined. To address this challenge, we study policy evaluation and learning under trunc...

Zi-Wei Huang, Zhi-Chao Jiang · 0 citations

Conformal Policy Learning with Distribution-Free Safety Guarantees

Conformal policy learning (CPL) is proposed, a policy learning procedure with a new distribution-free safety guarantee that controls the probability of assigning treatment to an individual who would be harmed relative to control and achieves asymptotically optimal welfare subject to the safety constraint.

Ying Jin, Naoki Egami · 1 citation
#machine learning Preprint Oct 2026

Variance-Optimal Off-Policy Evaluation with Conjunct Effect Modeling

Off-policy evaluation (OPE) for contextual bandit policies becomes challenging when action-level importance weighting incurs excessive variance. Doubly robust (DR) estimation remains unbiased under common support but retains these high-variance action-level weights. A prior estimator, Off-policy evaluation with Conjunc...

Nicolò Felicioni, Michael Benigni, Maurizio Ferrari Dacrema et al. · 0 citations
Review Sep 2026

Orthogonal Policy Learning with Ordinal Outcomes

Policy learning methods based on conditional average treatment effects can obscure subpopulation heterogeneity when applied to ordinal outcomes. We develop a policy learning framework for ordinal outcomes with heterogeneous utilities for individuals who strictly benefit from treatment and those who do not. Since the pr...

Yue Zhang, Shan-Shan Luo, Yan-peng He · 0 citations

Stable Policy Learning

This paper proposes a method for policy learning called policy-vote bagging, which learns treatment decisions on many subsamples then averages their votes into treatment probabilities, which preserves expected welfare and improves expected utility for a risk-averse researcher.

Harvey Barnhard, Giacomo Opocher, Rahul Singh · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.