Reviewer #1 (Public review): Error-driven representation learning in the mesolimbic system

Abstract

In reinforcement learning, an agent learns to map representations of the environment state to predictions of future reward. Most prior work in neuroscience has assumed a fixed representation and studied how reward prediction errors (thought to be conveyed by phasic dopamine signals) are used to update the mapping from representations to predictions. However, work in machine learning has demonstrated that much more powerful predictive systems can be learned by using the errors to update the representations themselves. We study whether the brain does something similar by leveraging simultaneous recordings of striatal projection neurons in the olfactory tubercle (putatively representing state features) and dopamine neurons in the ventral tegmental area. We show that trial-by-trial changes in striatal activity are more consistent with dopamine-driven representation learning than a variety of alternative updating schemes. This result suggests a convergence of representation learning principles in biological and artificial systems.