This is the first demonstration that a learned Hamiltonian can qualitatively extrapolate from predominantly regular dynamics into a broad chaotic sea absent from training, and what decides parameter extrapolation is not Hamiltonian structure alone but how the fitted Hamiltonian continues in the control parameter.
Abstract
Machine learning of Hamiltonian dynamics has driven growing interest in Hamiltonian neural networks (HNNs), which encode Hamilton's equations of motion into the learning architecture. Despite this progress, it remains unknown whether such networks can predict dynamical regimes absent from their training data, in particular the broad chaotic sea that emerges beyond the observed parameter interval. We address this question using a parameter-aware random-feature Hamiltonian neural network (RF-HNN). Trained using data from only a small number of control-parameter values at which invariant tori dominate, the RF-HNN predicts autonomous long-time dynamics at unseen parameter values where mixed phase space develops and chaotic regions expand, with no data from that regime used in training or model selection. The method is demonstrated across four two-degree-of-freedom Hamiltonian families, including the H\'enon-Heiles system. Using Poincar\'e-section geometry and finite-time Lyapunov exponents, we show that the RF-HNN reproduces the breakup of regular structures and the emergence and growth of chaotic regions, whereas conventionally trained HNNs with the same Hamiltonian structure remain too regular. These results show that what decides parameter extrapolation is not Hamiltonian structure alone but how the fitted Hamiltonian continues in the control parameter. To our knowledge, this is the first demonstration that a learned Hamiltonian can qualitatively extrapolate from predominantly regular dynamics into a broad chaotic sea absent from training.
Hamiltonian Neural Networks (HNNs) parameterize conservative dynamics through a learned scalar Hamiltonian, providing an architectural prior that is absent from generic vector-field neural networks. We evaluate this prior under a controlled protocol in which an HNN and a parameter-matched feedforward baseline are trained on the same RK4-generated trajectories, use the same central-difference derivative targets and optimization settings, and are integrated at inference with the same RK4 scheme. Results are reported over five independent training seeds. On the nonlinear pendulum, the HNN reduces mean energy drift by 42-fold and mean trajectory MSE by 15.8-fold at T = 100, approximately 16 pendulum periods. Its energy drift also remains bounded and exhibits substantially lower seed-to-seed variability than the standard-network baseline. An energy-stratified analysis shows that the difference becomes more pronounced as trajectories explore more nonlinear regions of phase space. As an additional diagnostic, we examine an explicit St\"ormer--Verlet-style rollout of the learned HNN. Because the learned Hamiltonian is not constrained to the separable form H(q,p) = T(p) + V(q), the standard symplecticity guarantee of velocity Verlet does not directly apply. We further apply the same matched-integrator protocol to the three-dimensional Kepler two-body problem. The HNN again exhibits lower trajectory, energy, and angular-momentum drift than the parameter-matched baseline. These experiments provide a controlled study of how Hamiltonian parameterization affects long-horizon prediction and physical consistency across two conservative dynamical systems.
Randomly connected neural networks undergo a transition from a stable fixed point to chaos as the coupling strength increases. In the thermodynamic limit, this transition has been shown theoretically to occur abruptly at a critical point. In finite-size systems, however, a variety of bifurcation cascades appear between the stable fixed point and chaos. In this study, we systematically characterize routes to chaos in finite-size random neural networks. By analyzing individual realizations, we identify multiple scenarios, including the Ruelle-Takens-Newhouse route, torus doubling, and fractalization, as well as chaotic dynamics with persistent toroidal geometry. We also study these behaviors at the ensemble level by quantifying the fraction of chaotic trajectories as a function of coupling strength and system size. The resulting finite-size crossover sharpens with increasing system size and exhibits empirical scaling trends, providing a statistical characterization of the broad onset of chaos in finite random networks.
Reliable finite-horizon forecasting of chaotic dynamics is challenging because small approximation errors grow rapidly during recursive prediction. This study presents a controlled comparison of data-driven and physics-regularized forecasting methods for the Lorenz and Rössler systems. The proposed Hybrid Physics-Informed Feedforward Neural Network (Hybrid PI-FNN) learns a discrete state-transition map from a ten-state observation window through a five-step recursive rollout. Unlike conventional continuous-coordinate physics-informed neural networks, physical consistency is imposed using fourth-order Runge–Kutta transition targets derived from the known governing equations. The physics weight is selected using chronological recursive validation and evaluated against an architecturally identical multi-step FNN with λ=0. Conventional FNN, LSTM, Echo State Network (ESN), Autoregressive AR(10), and Dynamic Mode Decomposition baselines are also evaluated using untouched test trajectories. For the 1000-step Lorenz test rollout, the ESN achieved the lowest mean squared error (MSE) of 0.1701, followed by the LSTM with 22.9962. The Hybrid PI-FNN produced an MSE of 95.8157, compared with 87.9260 for its λ=0 ablation; therefore, physics regularization did not improve Lorenz test MSE, although their forecast horizons at a 10% normalized-error threshold were similar (273 and 272 steps, respectively). For the Rössler system, the Hybrid PI-FNN reduced recursive MSE from 0.2202 for the λ=0 ablation to 0.0834, corresponding to a 62.11% reduction, while both models completed the maximum evaluated 1000-step forecast horizon. Nevertheless, the ESN again achieved the lowest Rössler MSE of approximately 8.04×10−5. Finite-horizon correlation-dimension analysis, exact governing-equation Lyapunov spectra, computational-cost comparisons, and five-seed paired experiments were additionally conducted. The exact spectra confirmed one positive, one approximately neutral, and one negative exponent for each system, indicating chaotic but not hyperchaotic behavior. The paired multi-seed analysis did not establish a statistically significant forecasting advantage from physics regularization. These findings show that higher-order physics consistency can benefit particular systems and configurations, but it does not guarantee universal superiority in recursive chaotic forecasting.
Abdul Karim, M. Carratù, In Cheol Jeong· Mathematics· 0 citations
Multi-step training of sparse, interpretable models of dynamical systems directly from time-series data yields models with accurate short-term dynamics and strong agreement in long-time statistical properties, including mean, variance, and Lyapunov exponents.
This study investigates the capability of deep neural networks to infer the time-evolution of the Rössler system a canonical chaotic oscillator by leveraging initial conditions and forcing parameters as input variables and underscores the need for hybrid approaches to address long-term instability.
A. Fateh, Harrag Abdelmalek, F. Mohamed et al.· International Journal of App...· 0 citations
Locating periodic solutions of chaotic dynamical systems normally requires an initial guess close enough to the target orbit for numerical continuation or gradient-based search to converge. We show that Physics-Informed Neural Networks (PINNs) trained on sparse, noisy observations \emph{without} initial conditions recover periodic orbits of the gravitational three-body problem, including orbit families absent from the training data. The method rests on a second-order ODE formulation, fixed-frequency Fourier features, percentile-based adaptive refinement, and a trainable scaling parameter, each validated on forward problems. Across two 100-seed ensembles, $23$--$25\%$ of runs converge to families not present in the training data. We then ask what determines which family emerges. Two $\chi^2$ tests give a consistent answer: changing the training data source significantly shifts the distribution of recovered families ($p<0.001$, Cram\'{e}r's $V = 0.339$), whereas switching between the two initialization distributions tested does not ($p = 0.620$, $V = 0.094$). The random seed selects which family a given run recovers; the \emph{distribution} the weights are drawn from does not shift the aggregate frequencies, but the training data does. The evidence is empirical: we do not characterize the loss landscape analytically, and PINNs remain slower than conventional integrators on well-posed initial-value problems. What the experiments establish is that the recovered orbits are verifiable rather than merely plausible: the identified ones refine to genuine periodic solutions, a network trained on Lagrange data recovers the figure-eight choreography (Li--Liao class I.A.1, matched to seven significant digits in $T^*$), and one trained on figure-eight data recovers a Broucke--Hadjidemetriou--H\'{e}non orbit closing to $\delta_T<10^{-9}$.
Nikolaos Kollias, NIKOLAOS M. Matzakos· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
