Biased processing of multiple outcomes in human reinforcement learning: evidence from computational modeling and eye-tracking

Abstract

In many circumstances, choices result in multiple simultaneous outcomes, all of which should be integrated to optimally update reward expectation. Yet, to date, empirical investigations of reinforcement learning have mostly focused on situations where choices deliver only one outcome at a time. To understand how humans learn from multiple outcomes, we designed a new reinforcement learning task, where the selection of an option resulted in two outcomes drawn from the same underlying distribution over potential gains and losses, even if ultimately only one of the two counted for the final payoff. Behavioral results show that the two - equally informative- outcomes are considered, yet asymmetrically as function of their valence. This behavioral observation is backed up by computational analyses showing that, on top of previously documented asymmetric update, multiple outcome integration is biased, such as it overweighs rewards over punishments. Behavioral and computational results were paralleled by eye-tracking analysis showing that attention deployment is biased by outcome valence and relevance. The main results were confirmed in a second experiment featuring complete feedback information. Overall, our findings suggest that, when options deliver multiple discordant outcomes, losses tend to be neglected compared to gains.