Policy regularization as a unifying theory of the striatal division of labor in learning

Abstract

Learning novel behaviors requires balancing previously learned actions with the ability to flexibly adapt to changing reward contingencies. This trade-off is well documented in the division of labor between dorsolateral striatum (DLS), which promotes selection of cached, history-dependent actions, and dorsomedial striatum (DMS), which supports flexible learning as reward contingencies change. Here, we propose policy regularization as a general computational principle for understanding this division. Capacity-limited agents face a fundamental trade-off between maximizing reward and minimizing the cost of deviating from a default policy that caches frequently used action transitions. We formalize this trade-off as a KL-regularized reward objective in which a flexible controller (DMS) incurs a cost for diverging from a history-dependent default (DLS). The resulting optimal policy is a combination of a reward-driven action value, continuously updated by DMS, and a default policy conditioned on action history, cached by DLS. In practice, DLS consolidates the action transitions shaped by DMS’s reward-driven value learning, so DLS preserves behaviors that were once optimal even when reward contingencies change—producing robust but inflexible action selection and effectively “regularizing” DMS-driven learning. We show that a single model—with one shared parameter set and consistent lesion rules—provides a unifying explanation for the functional organization of the striatum, reproducing canonical DLS–DMS dissociations in outcome devaluation, serial spatial reversal, skilled action sequencing, and motor sequence execution tasks. Significance Statement How does the brain balance old, well-practiced behaviors with the flexibility to adapt as reward contingencies change? We propose a theoretical framework in which a single principle—maximizing reward while minimizing the cost of deviating from previously consolidated action patterns— explains the division of labor in the dorsal striatum: the dorsomedial striatum learns reward-driven action values that support flexible, goal-directed behavior, while the dorsolateral striatum caches recurring action transitions regardless of current reward. A single model reproduces canonical lesion dissociations across four behavioral paradigms, offering a normative explanation for why these two subregions exist and compete, and providing a potential circuit implementation underlying learning.