DAW reshapes the loss landscape to allocate representational capacity toward the sparse, high-d regimes where forecast errors are systematically large, and consistently outperforms uniform training, purely statistical density weighting, and its randomly permuted ablation on the chaotic KS equation.
Abstract
Deep learning surrogates for forecasting chaotic dynamical systems suffer from catastrophic error accumulation over long-term autoregressive rollouts. This behavior is partly tied to the underlying systems: chaotic spatiotemporal systems, such as the Kuramoto-Sivashinsky (KS) equation, visit phase space unevenly - dominated by recurrent, low-dimensional quiescent states (e.g., near-laminar flows) and punctuated by rare, dynamically complex topological transitions (e.g., wave-merging events). Under a sample-wise uniform objective, standard neural surrogates allocate their finite capacity to the statistically numerous quiescent states, under-representing the transient regimes that trigger disproportionate, localized errors. Existing imbalanced-regression methods reweight samples by target-space density. However, statistical target-space rarity need not coincide with the intrinsic dynamical rarity - the recurrence geometry of the attractor that is the source of the imbalance. To address this, we introduce Dynamics-Aware Weighting (DAW), a data-centric objective reweighting framework. Using the local dimension $d$ from dynamical systems theory as an a priori measure of a state's active degrees of freedom, DAW reshapes the loss landscape to allocate representational capacity toward the sparse, high-$d$ regimes where forecast errors are systematically large. On the chaotic KS equation, DAW consistently outperforms uniform training, purely statistical density weighting, and its randomly permuted ablation, reducing long-term autoregressive error relative to all baselines. Event-level analysis shows that DAW achieves this by suppressing the localized error amplifications incurred during sharp jumps in $d$, which accompany complex physical processes such as wave-merging in the KS system.
This study investigates the capability of deep neural networks to infer the time-evolution of the Rössler system a canonical chaotic oscillator by leveraging initial conditions and forcing parameters as input variables and underscores the need for hybrid approaches to address long-term instability.
A. Fateh, Harrag Abdelmalek, F. Mohamed et al.· International Journal of App...· 0 citations
Reliable finite-horizon forecasting of chaotic dynamics is challenging because small approximation errors grow rapidly during recursive prediction. This study presents a controlled comparison of data-driven and physics-regularized forecasting methods for the Lorenz and Rössler systems. The proposed Hybrid Physics-Informed Feedforward Neural Network (Hybrid PI-FNN) learns a discrete state-transition map from a ten-state observation window through a five-step recursive rollout. Unlike conventional continuous-coordinate physics-informed neural networks, physical consistency is imposed using fourth-order Runge–Kutta transition targets derived from the known governing equations. The physics weight is selected using chronological recursive validation and evaluated against an architecturally identical multi-step FNN with λ=0. Conventional FNN, LSTM, Echo State Network (ESN), Autoregressive AR(10), and Dynamic Mode Decomposition baselines are also evaluated using untouched test trajectories. For the 1000-step Lorenz test rollout, the ESN achieved the lowest mean squared error (MSE) of 0.1701, followed by the LSTM with 22.9962. The Hybrid PI-FNN produced an MSE of 95.8157, compared with 87.9260 for its λ=0 ablation; therefore, physics regularization did not improve Lorenz test MSE, although their forecast horizons at a 10% normalized-error threshold were similar (273 and 272 steps, respectively). For the Rössler system, the Hybrid PI-FNN reduced recursive MSE from 0.2202 for the λ=0 ablation to 0.0834, corresponding to a 62.11% reduction, while both models completed the maximum evaluated 1000-step forecast horizon. Nevertheless, the ESN again achieved the lowest Rössler MSE of approximately 8.04×10−5. Finite-horizon correlation-dimension analysis, exact governing-equation Lyapunov spectra, computational-cost comparisons, and five-seed paired experiments were additionally conducted. The exact spectra confirmed one positive, one approximately neutral, and one negative exponent for each system, indicating chaotic but not hyperchaotic behavior. The paired multi-seed analysis did not establish a statistically significant forecasting advantage from physics regularization. These findings show that higher-order physics consistency can benefit particular systems and configurations, but it does not guarantee universal superiority in recursive chaotic forecasting.
Abdul Karim, M. Carratù, In Cheol Jeong· Mathematics· 0 citations
Chaotic time series forecasting is a challenging task due to its sensitivity to initial conditions and long-term unpredictability. Traditional methods typically rely on sufficient temporal trajectories to learn long-term dynamics, which limits their applicability when only short-term observations are available. While recent Large Language Models (LLMs) have shown great potential for time series forecasting, their temporal representations are not explicitly tailored to the phase-space structure and nonlinear evolution of chaotic systems. To address these issues, we propose PAC-LLM, a phase-space-aware adaptive fusion framework for long-term chaotic time series forecasting powered by LLMs. PAC-LLM leverages learned phase-space features and textual information to fully enable LLM's time series forecasting capacity. In particular, we design an auxiliary feature module and a gated weighting mechanism for multivariate coupling information fusion and selection. Extensive experiments on representative chaotic systems demonstrate that our method outperforms existing fine-tuned and zero-shot baselines in both short-term and long-term predictions. Our ablation study further confirms the effectiveness of each key component in PAC-LLM.
Multi-step training of sparse, interpretable models of dynamical systems directly from time-series data yields models with accurate short-term dynamics and strong agreement in long-time statistical properties, including mean, variance, and Lyapunov exponents.
Deploying machine learning surrogates in scientific simulations faces multifaceted challenges, primary among which is the lack of Continual Learning (CL) capabilities—specifically, the inability to adapt to new physical regimes without significantly degrading performance on prior ones. This is particularly problematic for autoregressive surrogates of time-dependent Partial Differential Equations (PDEs), where small prediction errors can accumulate over long rollouts and new physical regimes overwrite previously learned dynamics. We formulate this adaptation as a CL problem, demonstrating that while standard Experience Replay (ER) is a robust baseline across Advection-Diffusion, Burgers’, and Navier-Stokes equations, storing full high-resolution rollouts can be memory-inefficient. To address this, we introduce Replay-TS, a temporal-slicing replay strategy that stores compact autoregressive windows sampled across past simulations. Through empirical analysis, we show that Replay-TS exploits the low-frequency spectral redundancy of physical systems to enable sparse supervision for rollout steps. By preserving the contiguous historical context and sparsely penalizing the autoregressive target steps, Replay-TS improves retention performance under a fixed memory budget by leveraging higher sample diversity. Replay-TS consistently outperforms standard ER methods across standard 1D and 2D streams, achieving over a 30% MSE reduction in a mixed-physics stream, while remaining architecture-agnostic.
Hamed Hemati, Binh Duong Nguyen, Stefan Sandfeld· Machine Learning for Computa...· 0 citations
This work introduces a probabilistic, non-intrusive reduced-order model (ROM) for chaotic dynamical systems, arguing that projecting high-dimensional nonlinear dynamics onto a low-dimensional manifold introduces irreducible uncertainty, compounded by the chaotic attractors and multi-admissible futures inherent to turbulent flows.
Ismaël Zighed, Nicolas Thome, Patrick Gallinari et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
