Discrete synaptic states and context-modulated readouts support continual learning

Abstract

Biological neural systems learn new behaviors while retaining earlier ones, whereas sequentially trained artificial networks often overwrite parameters and forget previously learned behaviors. We introduce the Context-modulated Synaptic-states Recurrent Neural Network (CoSyn-RNN), a new continual-learning model inspired by heterogeneous synaptic states and neuro-modulatory control. CoSyn-RNN combines sparse recurrent allocation with a discrete protection rule: after each task, recurrent neurons whose synaptic weights cross a fixed threshold form a task-specific neuron group, and their associated parameters enter a protected state during later learning. A neuron-specific gain and a task-cued mask over a fixed base readout control how recurrent activity contributes to behavior. Across various task sequences consisting of many cognitive tasks, training and validation losses and accuracies were maintained throughout learning. The protection rule produced modular, task-ordered recurrent structure compatible with forward reuse of earlier computations. Pairwise recruitment measurements further defined minimum- and maximum-cost curricula that produced markedly different final recurrent structures and different remaining capacity for future learning. Thus, CoSyn-RNN provides a biologically inspired model in which protected synaptic states, sparse allocation, and context-dependent modulation support continual learning.