A perturbation-based learning rule for online self-supervised learning in ESNs is proposed, derived from an orthogonal decomposition of the self-supervised learning cost, which separates an input-dependent component from a redundant component determined by the fixed ESN parameters.
Abstract
Intelligent systems should not only solve tasks but also adapt under real-world constraints. Autonomous adaptation via self-supervised learning, sequential adaptation via online learning, and memory-efficient implementation via perturbation-based learning are important requirements for such systems. However, these requirements are generally in tension for high-dimensional systems, because perturbation-based learning suffers from variance that grows with the dimension of the perturbed variables. In this study, we focus on echo state networks (ESNs), where this tension naturally arises in large reservoirs. We propose a perturbation-based learning rule for online self-supervised learning in ESNs. The proposed rule is derived from an orthogonal decomposition of the self-supervised learning cost, which separates an input-dependent component from a redundant component determined by the fixed ESN parameters. By perturbing only the input-dependent component, the effective perturbation dimension is reduced from the reservoir dimension to the input dimension. Thus, the proposed method preserves self-supervised adaptation, online learning, and scalar-feedback perturbation learning, while avoiding reservoir-size-dependent variance growth. This suggests a design principle for scalable and hardware-compatible learning: online learning should be restricted to the dynamically necessary low-dimensional component of the objective.
Recent work has identified incremental learning in shallow networks trained on single-index and multi-index models. However, existing analyses often rely on simplifying settings, such as small initialization, correlation loss, or layer-wise training. These choices reduce neuron interactions and leave some feature learning dynamics under standard initialization unexplored. We study training dynamics for polynomial-width two-layer networks learning orthogonal multi-index targets under standard initialization using polynomially many samples. We first prove that incremental learning still occurs: the loss decreases sequentially according to the Hermite expansion of the target, with lower-order components learned before higher-order components recover the individual target directions. In this standard initialization regime, training also shows a competitive reallocation of parameter mass: after the total mass fits the target mean and stabilizes, mass shifts into the target subspace and then concentrates on aligned neurons. Our theoretical analysis uses slightly modified gradient flow, while vanilla gradient descent empirically exhibits the same qualitative dynamics. Technically, we introduce a symmetry-based finite-width approximation via symmetrized networks, rather than comparing directly with an infinite-width limit. This yields better control of approximation errors and may be of independent interest.
Mo Zhou, Weihang Xu, Simon S. Du et al.· 0 citations
The activity of large neuronal populations is often confined to low-dimensional manifolds that can drift over time, posing a challenge for learning rules that assume stable, full-rank representations. Here, we introduce SPLiT (Synaptic Projection Learning with intrinsic Tracking), a synaptic plasticity rule for recurrent neural networks that combines unsupervised manifold tracking with supervised learning. SPLiT uses an online Oja rule to continuously estimate the intrinsic low-dimensional activity subspace and performs normalized least-mean-squares learning in manifold coordinates. In this way, SPLiT keeps synaptic weights aligned with evolving population dynamics. Using a rate-based recurrent network, we show that SPLiT reliably learns time-varying target signals under both constrained dynamics, where activity is restricted to a fixed low-dimensional manifold, and unconstrained dynamics exhibiting changes in the dominant activity subspace. We show that manifold-constrained learning with SPLiT yields faster convergence, less sensitivity to recurrent gain, smaller weight updates, and is more robust to noisy teaching signals. Analytical results show that SPLiT learns the optimal decoder projected onto the instantaneous principal subspace and maintains bounded error under drift. Together, these findings provide a mechanistic account of how synaptic plasticity can leverage the structure of neural manifolds to enable stable and efficient supervised learning despite representational drift.
Online class-incremental learning is a challenging setting in which the model continually learns new classes from a single pass over the data while retaining discrimination among previously seen classes. Although exemplar replay performs well in this setting, it incurs substantial memory and computational cost. Prototype replay offers a compact alternative by storing a single representative feature vector per class, but in cold-start settings it faces a stability-plasticity dilemma: limited pretraining can reduce the usefulness of the learned representation for downstream tasks, while continued adaptation through finetuning can disrupt the stability of the feature space and undermine the reliability of stored prototypes. To address this dilemma, we propose a continual learning mechanism that combines prototype replay with a feature extractor enhanced with activity-dependent metaplasticity in a spiking network. During online learning, metaplasticity selectively reduces the plasticity of important synaptic weights, limiting feature drift while preserving adaptability in the feature extractor. This is followed by a sleep phase in which stored class prototypes are replayed to improve discrimination across all seen classes. Evaluations on two continual learning benchmarks show that the proposed approach achieves a more favorable stability-plasticity tradeoff than either fully freezing or fully finetuning the feature extractor, while providing more favorable replay memory scaling than exemplar feature replay.
F. Zohora, D. Kudithipudi· International Conference on...· 0 citations
Deep neural networks are high-dimensional dynamical systems whose function depends on the stable propagation of activity and perturbations across many layers. Maintaining suitable dynamical regimes may therefore be essential for robust learning and for preventing dynamical instabilities during training. Here, we show that the global dynamical state of a deep neural network can be autonomously regulated by purely local homeostatic plasticity. Neuronal activity is inferred from responses across inputs, and individual synapses are strengthened or weakened using only the activity of their postsynaptic neuron. Without measuring any global network property, this rule drives networks from both subcritical and supercritical initial conditions toward a common critical state, characterized by conserved activity propagation and a vanishing largest finite-time Lyapunov exponent. When combined with gradient-based learning, homeostatic adaptation counteracts the training-induced drift toward supercritical dynamics, while revealing a competition between dynamical regulation and task optimization. Our results demonstrate how adaptive self-organization can be implemented in deep neural networks and how local plasticity can control their collective dynamical operating point.
Adaptive optimization algorithms are fundamental to modern deep learning; however, the global organization of neural-network training regimes induced by optimizer hyperparameters remains insufficiently understood. In particular, the influence of the Adam moment coefficients on the stability and qualitative behavior of the learning process has not been systematically investigated through parameter-space regime mapping. In this work, we introduce an observable-based empirical regime-mapping framework for analysing neural-network training under the Adam optimizer in the two-dimensional hyperparameter space defined by the exponential decay coefficients of the first and second gradient moments, (β
1
, β
2
). Neural-network optimization is treated as an iterative parameter-update process evolving in a high-dimensional parameter space, while its behavior is characterized through low-dimensional observable fields derived from neuron-wise training-error dynamics. Rather than relying on a single observable, the proposed framework combines seven complementary empirical descriptors that characterize regime organization, temporal stability, alignment, anisotropy, and training evolution. Experiments are performed on datasets of increasing complexity, including printed-digit patterns, Fashion-MNIST, CIFAR-10, and CIFAR-100, using multilayer neural-network architectures of varying width and depth. The resulting regime maps reveal fragmented hyperparameter landscapes containing stable, oscillatory, slow-learning, and irregular observable regimes. Increasing dataset complexity and network capacity is generally associated with smaller coherent stable regions and increased sensitivity to the Adam moment coefficients. The geometric complexity of the regime boundaries is quantified using box-counting analysis. The estimated dimensions approach D ≈ 1.9 for several investigated configurations, indicating highly irregular and nearly space-filling boundaries at the available numerical resolution. These values are interpreted as empirical measures of boundary complexity rather than as evidence of exact mathematical fractality. Additional robustness experiments performed using 300, 500, and 1000 optimizer steps demonstrate that the large-scale organization of all seven observable fields remains largely preserved, whereas the principal changes are concentrated near transition boundaries. This persistence indicates that the detected regime structures are reproducible and are not solely artifacts of short optimization histories. The proposed methodology provides an empirical computational framework for visualizing and comparing optimization regimes in the Adam hyperparameter space. It facilitates the identification of comparatively stable hyperparameter regions and offers a complementary observable-based perspective on the complex behavior of adaptive neural-network optimization.
S. Sveleba, I. Katerynchuk, I. Kunyo et al.· Neural Processing Letters· 0 citations
Batch normalization (BN) substantially improves sample efficiency in continuous-control actor-critic methods such as CrossQ, yet recent studies report performance degradation in discrete-action value learning on Atari. These failures are surprising because discrete Q-networks lack the action-input distribution mismatch identified by CrossQ. We show for target-based C51 and target-free PQN that the simple choice between running and batch statistics at specific forward passes can reverse this degradation. In C51, switching the BN bootstrap forward to batch-statistic mode significantly improves performance over unnormalized and LayerNorm baselines and scales stably with update-to-data ratios up to 12. In PQN, using batch-statistics for both action selection and bootstrapping recovers performance from the failing running-statistic configuration. Across 26 Atari games at 400M frames, this configuration achieves a higher final aggregate score than PQN with LayerNorm. Our results show that carefully configured BN can substantially improve discrete-action value learning, and that its forward protocols are an essential part of the algorithm specification.
Daniel Palenicek, Mikael Henaff, Scott Fujimoto et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
