Spectral submanifolds can be used to reduce recurrent neural networks to low-dimensional models, revealing their core dynamics, and authors uncover robust structures underlying decision-making and working-memory tasks, providing predictions about the underlying behavior of neural computations.
Abstract
Neural ordinary differential equations (ODEs) are widely used in neuroscience to model the collective activity of neurons during behavioral tasks. The high dimensionality of their parameter and activity spaces, however, often make it challenging to infer and interpret the fundamental features of their dynamics. In this study, we employ recent nonlinear dynamical system techniques to uncover the core dynamics of several Neural ODEs used in contemporary neuroscience. Specifically, using a data-driven approach, we identify Spectral Submanifolds (SSMs), i.e., low-dimensional attracting invariant manifolds tangent to the eigenspaces of fixed points. The internal dynamics of SSMs serve as nonlinear models that reduce the dimensionality of the full RNNs by orders of magnitude. Through low-dimensional, SSM-reduced models, we give mathematically precise definitions of line and ring attractors, which are intuitive concepts commonly used to explain decision-making and working memory. This unprecedented level of understanding of Neural ODEs obtained from SSM reduction enables the interpretation of mathematically well-defined and robust structures in neuronal dynamics, leading to predictions about the neural computations underlying behavior. Spectral submanifolds can be used to reduce recurrent neural networks to low-dimensional models, revealing their core dynamics. Here, authors uncover robust structures underlying decision-making and working-memory tasks, providing predictions about the underlying behavior of neural computations.
Multi-step training of sparse, interpretable models of dynamical systems directly from time-series data yields models with accurate short-term dynamics and strong agreement in long-time statistical properties, including mean, variance, and Lyapunov exponents.
Data-driven modelling of nonlinear dynamical systems based on machine learning is reviewed in this paper. Nonlinear dynamics refer to all sorts of complicated and irregular phenomena in mathematics, physics, engineering and life that are not well explained by linear models. Based on the accumulation of observation data from sensors, experiments and high-fidelity simulations, machine learning has recently begun to be applied in system identification, state estimation and the discovery of governing equations. This paper systematically presents the basic ideas of dynamical systems and then introduces several data-driven methods, such as Sparse Identification of Nonlinear Dynamics (SINDy), Neural Ordinary Differential Equations (Neural ODEs) and Koopman operator theory. Based on the above research, a number of methods have been proposed to solve the problems of noise, irregular sampling, etc., in high-dimensional chaos, and their respective advantages are introduced below. At present, the problems of poor data quality, partial observability and lack of interpretability in deep learning models are well-known. In the future, Physics-Informed Machine Learning, uncertainty quantification and robust model methods will be applied in engineering. Data science and applied mathematics will be combined in this paper to offer a comprehensive introduction to the problems and models of complex systems.
Ming-Yang Wang· Theoretical and Natural Scie...· 0 citations
An interpretable identification framework based on universal neural ordinary differential equations (UNODEs), symbolic regression, and parameter refinement is developed that is competitive on autonomous polynomial systems and more effective in recovering compact symbolic structures for non-polynomial and explicitly time-varying dynamics.
Qing-Tong Dong· Engineering Research Express· 0 citations
Recurrent Neural Networks (RNNs) are widely used to model neural activity in Computational Neuroscience. Here, we explore the mathematical foundations of three fundamental procedures that can be implemented: temporal rescaling, discretization, and linearization. These techniques provide crucial tools for characterizing the behavior of RNNs, offering insights into their temporal dynamics, facilitating practical computational implementation, and allowing for linear approximations for analysis. We discuss the flexible order in which these procedures can be applied, emphasizing their importance in modeling and analyzing RNNs for neuroscience and formally prove that these three operations commute pairwise. We also explicitly describe the conditions under which these procedures can be considered interchangeable. Our findings directly inform the design of biologically plausible RNN models for simulating neural dynamics observed in decision-making circuits and motor control, where temporal scaling and stability are critical for matching experimental recordings. Furthermore, we show that this exact commutativity guarantees the structural preservation of the network's controllability, preventing the emergence of inaccessible state-spaces under numerical discretization or temporal rescaling.
The Dynamic Mode Decomposition (DMD) has been consolidated as a basic tool for data-driven analysis of dynamical systems, allowing simultaneous identification of coherent structures and their dynamics from time-resolved measurements. However, with a linear regression at its core, DMD is unable to produce accurate models from recordings of dynamics that are inherently nonlinear, such as the response to large perturbations and the evolution on chaotic attractors. Recent approaches attempt to simultaneously fit the linear and nonlinear contributions to the dynamics by performing a regression onto a physically motivated model structure. However, although the resulting nonlinear models can produce accurate short-term predictions, their linearization does not necessarily agree with that of the original system. In this work, we introduce a novel data-driven method --- nonlinearity-subtracted DMD (NSDMD) --- that focuses on producing an accurate linearization of a system when the nonlinear contribution to its dynamics are available while the linear part is not. This scenario is encountered, for example, when the nonlinear terms in the governing equations are known, while the linear operator contains uncertain material properties or it accounts for the closure of unresolved dynamics. This also arises when the data is generated by a black-box simulation code that is able to output the nonlinearity, but not the action of the linear operator on the snapshots. NSDMD leverages data snapshots of the nonlinearity to explicitly account for the purely nonlinear contributions to the dynamics and formulate a regression problem that finds a low-rank approximation of the underlying linear operator. We demonstrate the approach on several numerical examples, showcasing its improved capabilities for data-driven linear analysis of chaotic, partially observed, advection-dominated, and high-dimensional dynamics.
Benjamín Herrmann, Katherine Cao, S. Brunton et al.· 0 citations
The problem of sparse identification of non-linear dynamics is considered. The problem is motivated by the need for interpretable mathematical models in the natural sciences, where the fundamental laws of evolution are either unknown or known only partially.
The T‑SINDy method is proposed, which combines time delay embedding, tensor representations, and sparse regression. In contrast to the classical SINDy approach, in which the number of parameters grows exponentially with the number of candidate functions, the proposed tensor map enables parameterization of all possible nonlinear interactions. To reduce computational complexity, canonical decomposition of rank R is used, reducing the number of parameter. Sparsity of the model is achieved through a two‑stage procedure: thresholding of the factor‑matrix elements followed by fine‑tuning of the nonzero coefficients, and then additional truncation of small entries in the unfolded parameter tensor.
Computational experiments are performed on the Lorenz system (with two of three variables observed) and on the normal form of the Hopf bifurcation (with a single observed variable) under noise levels ranging from 0 to 10%. It is demonstrated that T‑SINDy provides prediction accuracy comparable to that of the classical SINDy method while reducing training time. The reconstructed equations retain interpretability, explicitly expressing the dynamics in terms of the original variables, which constitutes a distinct advantage over neural‑network‑based methods. The proposed approach offers an efficient and interpretable alternative for the identification of dynamical systems from incomplete, noisy observations.
Denis M. Tikhonov, V. Strijov· Modeling and Analysis of Inf...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
