The results show that reservoir computing can be designed using brain properties and theoretical insights borrowed from the physics of forced nonlinear oscillators, and the frequency-based approach can be optimized to improve short-term prediction, a property that random reservoirs lack.
Abstract
Reservoir computing has emerged as an efficient machine learning framework for predicting time series generated by dynamical systems. In contrast to other machine and deep learning approaches, a reservoir computing trains only the output layer via linear regression, leaving the reservoir (recurrent layer) untrained. This simplification makes reservoir computers easier to train and more amenable to experimentation. However, because current reservoirs consist of networks of randomly connected nodes and require the optimization of numerous hyperparameters, a framework that precisely explains how reservoir computing operates and how it can be optimized remains missing. Here, we propose a frequency-based reservoir inspired by the brain's oscillatory dynamics and its hierarchy of timescales. The frequency-based reservoir can be interpreted as an ensemble of independent oscillatory units, each processing a portion of the input's frequency content. This allows us to understand the reservoir's internal behavior by modeling it as a single unit driven by an external input. Borrowing from the theory of a nonlinear oscillator forced by complex periodic inputs, we found that units of the frequency-based reservoir selectively amplify and store specific input frequencies, which are then used for prediction. The frequency-based reservoir performs as well as or better than equivalent random reservoirs. Furthermore, the frequency-based approach can be optimized to improve short-term prediction, a property that random reservoirs lack. Finally, we show that the frequency-based reservoir can also predict complex spatiotemporal dynamics. Our results show that reservoir computing can be designed using brain properties and theoretical insights borrowed from the physics of forced nonlinear oscillators.
Abstract.
Reservoir computing (RC) is a machine learning framework based on recurrent neural networks, which can naturally be viewed as dynamical systems. We focus on the problem of learning a time series generated by an unknown dynamical system [Formula: see text]. As suggested by several numerical studies, once the reservoir has learned [Formula: see text], it appears to reproduce the dynamics of [Formula: see text]; however, the underlying mechanism behind this behavior has not yet been fully clarified. In this study, we prove that, under certain assumptions, a reservoir that has learned [Formula: see text] becomes topologically semiconjugate in a weak sense or topologically conjugate to [Formula: see text]. This theorem and its proof shed new light on the mathematical foundations of RC.
We propose Tree Tensor Network Reservoir Computing (TTN-RC), a quantum-inspired reservoir computing framework for time-series prediction that uses the hierarchical structure of Tree Tensor Networks as a random reservoir. To control the exponential concentration or divergence of TTN outputs, we introduce a hierarchical ensemble method that partitions a fixed-size reservoir into multiple independent sub-reservoirs. In the tested NARMA benchmarks, TTN-RC achieves competitive or improved performance compared with conventional Echo State Networks, especially for tasks requiring higher-order nonlinear processing and longer contextual dependence. We also derive an expected contraction rate based on the reservoir Jacobian and develop a mean-field description of the reservoir-state statistics. These analyses identify an asymptotic stability boundary at $\sigma_{T}=\sqrt{2}$ in the large per-tree-size limit, where several theoretical indicators converge. Our results provide a design principle for tensor-network-based reservoir computing and clarify how hierarchical reservoir topology controls stability and nonlinear information processing.
Daiki Sasaki, Chih-Chieh Chen, T. Sogabe· 0 citations
A multi-regime RC framework in which multiple readouts are trained under different dynamical conditions and combined through a short observation window to form a trajectory-dependent linear readout enables both regime identification and adaptation to unseen or intermediate dynamics.
Sadegh Hadipour Lakmesari, H. Kantz, Francesco Sorrentino· Chaos· 0 citations
Machine learning methods are increasingly used for traffic prediction in applications such as autonomous driving. Such predictions must be both highly accurate and immediately available, making methods with low computational costs and fast training times of interest. One such method is reservoir computing, in which the rich dynamics of a nonlinear system serves as a computational substrate and only a linear readout vector is trained. In this work we use a traffic network as the reservoir for predicting the behavior of an undersensed traffic network. This matching of the highly nonlinear dynamics allows for similar encoding between the behaviors of the reservoir and target network, enabling a more direct prediction. We show that a reservoir governed by the Improved Intelligent Driver Model (IIDM) satisfies the echo state property for a class of slowly-varying inputs. Through simulations we show that the echo state property likely holds for a larger class of inputs, and that the IIDM reservoir computer (IIDM-RC) accurately predicts an undersensed vehicle network governed by varying car-following models. We also compare with echo state networks (ESNs) and Long Short-Term Memory (LSTM) networks, finding improvements using IIDM-RC in both prediction accuracy and training time.
Michael McCreesh, Rohit Gupta, Stephen L. Smith· arXiv.org· 0 citations
Reservoir computing exploits nonlinear dynamical systems to encode temporal inputs into high-dimensional state space representations. Although reservoir performance is often characterized through memory, nonlinearity, and their tradeoff, such aggregate measures do not reveal how task-relevant information is organized within the state space. Here, we introduce an eigen-spectral decomposition framework linking the degree-wise information processing capacity to the corresponding state space modes. As a result, we are able to quantify the degree-wise representation energy, and show that in some cases, substantial amounts of information processing capacity may reside in low-energy modes that are vulnerable to experimental noise. These results suggest that useful reservoir computation depends not only on dimensionality expansion, but also on the geometric organization of task-relevant information, with direct implications on physical reservoir computers.
Reservoir computing (RC) couples a fixed recurrent dynamical system with a trained lightweight readout, but this efficiency is partly lost during hyperparameter selection: the recurrent gain, input scale, and leakage rate determine the reservoir's stability and temporal processing regime and are usually tuned through many rollouts. We introduce a deterministic, pilot-informed selector for leaky linear reservoirs followed by coordinate-wise nonlinear features. Free probability yields cross-lag propagation coefficients that summarize how the reservoir mixes past inputs. In the large-width limit, these coefficients define a deterministic temporal kernel that approximates the finite-reservoir feature geometry. Kernel ridge regression on a short labelled pilot sequence therefore ranks candidate operating regimes without instantiating or rolling out a reservoir, and the selected configuration transfers across widths. Across ten synthetic temporal benchmarks, zero-rollout selection obtains a mean deployment score of $0.772$, compared with $0.774$ for exhaustive simulation-based search, while avoiding $156\,600$ selection rollouts. With a small rollout budget, the proposed ranking provides the strongest mean performance at every tested budget and reaches the exhaustive reference using $4.8\%$ of its rollout cost. On four public electricity-transformer-temperature (ETT) forecasting datasets, five retained candidates recover the exhaustive operating point on three datasets. On multivariate cellular-traffic forecasting, 15 rollouts per cell reach the 462-rollout exhaustive reference and outperform random search and Bayesian optimization at low budgets. These results position free-probability kernels as deterministic surrogates for selecting reservoir operating regimes when validation rollouts are scarce.
Sara Malacarne, Andrea Ceni, Claudio Gallicchio· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
