This paper proposes an ML-based approach to search for structure in dynamical systems using Neural Input Optimization (NIO), and discusses the methodology and presents promising results.
Abstract
The world consists of many dynamical and chaotic systems whose underlying functions can be highly nonlinear. Traditional engineering and scientific methods emphasize simplifying problems, sometimes to the point that valuable information may be lost. Instead, understanding of these systems may require analysis of the underlying dynamics. In the past, the technique of discovering and analyzing strange attractors has yielded some success. However, much work is still needed. Machine learning (ML)-based models such as deep neural networks which are based on nonlinear functions may provide a great set of techniques to help in the discovery of structure in dynamical systems. In this paper, we propose an ML-based approach to search for structure in dynamical systems using Neural Input Optimization (NIO). We discuss our methodology and present promising results. Quantitative results help to validate our proposed NIO methodology for structure discovery.
This study investigates the capability of deep neural networks to infer the time-evolution of the Rössler system a canonical chaotic oscillator by leveraging initial conditions and forcing parameters as input variables and underscores the need for hybrid approaches to address long-term instability.
A. Fateh, Harrag Abdelmalek, F. Mohamed et al.· International Journal of App...· 0 citations
Multi-step training of sparse, interpretable models of dynamical systems directly from time-series data yields models with accurate short-term dynamics and strong agreement in long-time statistical properties, including mean, variance, and Lyapunov exponents.
Data-driven modelling of nonlinear dynamical systems based on machine learning is reviewed in this paper. Nonlinear dynamics refer to all sorts of complicated and irregular phenomena in mathematics, physics, engineering and life that are not well explained by linear models. Based on the accumulation of observation data from sensors, experiments and high-fidelity simulations, machine learning has recently begun to be applied in system identification, state estimation and the discovery of governing equations. This paper systematically presents the basic ideas of dynamical systems and then introduces several data-driven methods, such as Sparse Identification of Nonlinear Dynamics (SINDy), Neural Ordinary Differential Equations (Neural ODEs) and Koopman operator theory. Based on the above research, a number of methods have been proposed to solve the problems of noise, irregular sampling, etc., in high-dimensional chaos, and their respective advantages are introduced below. At present, the problems of poor data quality, partial observability and lack of interpretability in deep learning models are well-known. In the future, Physics-Informed Machine Learning, uncertainty quantification and robust model methods will be applied in engineering. Data science and applied mathematics will be combined in this paper to offer a comprehensive introduction to the problems and models of complex systems.
Ming-Yang Wang· Theoretical and Natural Scie...· 0 citations
Abstract.
Reservoir computing (RC) is a machine learning framework based on recurrent neural networks, which can naturally be viewed as dynamical systems. We focus on the problem of learning a time series generated by an unknown dynamical system [Formula: see text]. As suggested by several numerical studies, once the reservoir has learned [Formula: see text], it appears to reproduce the dynamics of [Formula: see text]; however, the underlying mechanism behind this behavior has not yet been fully clarified. In this study, we prove that, under certain assumptions, a reservoir that has learned [Formula: see text] becomes topologically semiconjugate in a weak sense or topologically conjugate to [Formula: see text]. This theorem and its proof shed new light on the mathematical foundations of RC.
Hierarchical neural networks are widely used in artificial intelligence, yet their mathematical properties remain incompletely understood. In the infinite-width limit, two different theoretical frameworks have been proposed. One reduces deep learning to kernel regression with a fixed kernel by assuming that the parameters remain close to their initialization, whereas the other allows the parameters to move away from their initialization, requiring the kernel itself to be optimized. In this paper, we study a three-layer neural network with a finite but large number of hidden units. We show that training the input-to-hidden weights yields a smaller generalization error than keeping them fixed. Furthermore, the latter setting exhibits singularities in the parameter space, whereas the former does not. These findings indicate that singularities play an essential role even in wide neural networks.
Spectral submanifolds can be used to reduce recurrent neural networks to low-dimensional models, revealing their core dynamics, and authors uncover robust structures underlying decision-making and working-memory tasks, providing predictions about the underlying behavior of neural computations.
A. Marraffa, R. Krause, V. Mante et al.· Nature Communications· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
