Observational Multiplicity
A measure of regret is introduced for probabilistic classification tasks, which measures how the predictions of a model could change as a result of different training labels, and is used to show that regret is often higher for certain groups in the dataset and discuss potential applications of regret.