Skip to content
Preprint

Ensemble generative filtering for sequential data assimilation in dynamical systems

Sep 2026 · 2 citations · 61 references
Computer Science

TL;DR

The ensemble generative filter (EnGF) is proposed, a simple yet effective method for non-Gaussian filtering that delivers clear and often substantial improvements over the EnKF, in some cases even approaching the filtering accuracy of a massive-ensemble particle filter at a small fraction of its cost.

Abstract

Sequential data assimilation (DA) faces a fundamental trade-off: particle filters capture non-Gaussian cycling priors but require prohibitively large ensembles, whereas the ensemble Kalman filter (EnKF) is computationally efficient but constrained by its Gaussian assumption. As machine learning enables rapid model forecasts, exploiting non-Gaussian prior features via moderately large ensembles has become increasingly viable. To exploit this opportunity, we propose the ensemble generative filter (EnGF), a simple yet effective method for non-Gaussian filtering. The key idea is to fit a generative model to the forecast ensemble at each DA cycle and harness its defining strength, inexpensive sampling, to draw a much larger particle population for Bayesian analysis without any additional model forecasts; we adopt a Gaussian mixture model as a lightweight instance that can be fit cheaply from a moderate ensemble. To address practical challenges, we further extend the EnGF by introducing (i) a tempered EnGF using likelihood tempering to prevent particle degeneracy under informative observations and (ii) a latent EnGF that performs prior modeling and Bayesian updates in a reduced latent space for high-dimensional systems. Across chaotic systems (doubling map, Lorenz-63, and Lorenz-96) and a challenging shock-tube problem, the EnGF delivers clear and often substantial improvements over the EnKF, in some cases even approaching the filtering accuracy of a massive-ensemble particle filter at a small fraction of its cost.

View source

Similar papers

#machine learning Preprint Oct 2026

A Unified Framework for Bayesian Data Assimilation with Generative Models and Observation Interpolants

Bayesian data assimilation combines model forecasts with noisy observations, but sampling high-dimensional, non-Gaussian posteriors remains challenging. We introduce an observation-interpolant framework that turns pretrained stochastic interpolant, flow matching, and diffusion models into posterior samplers without ret...

N. Mücke, Benjamin Sanderse · 0 citations
#machine learning Preprint Sep 2026

AECSF: Adaptive Ensemble Conditional Score Filtering for High-Dimensional Nonlinear Data Assimilation

A training-free adaptive ensemble conditional score filter is proposed, which recasts noisy posterior score estimation as estimating the conditional mean of the system state given a noisy reverse particle and the observation, and establishes a bound relating conditional-mean estimation errors to reverse-sampling endpoi...

Yang-Wen Zhang, Shi-Wei Ni, Xiao-Ping Zhang et al. · 0 citations
Oct 2026

A localized weighted ensemble adjustment Kalman filter for high-dimensional nonlinear data assimilation

Particle filters provide a nonlinear Bayesian framework for data assimilation, but global weights often degenerate in high-dimensional geophysical systems. Ensemble Kalman filters are stable with practical ensemble sizes, yet their Gaussian and locally linear updates can be restrictive for nonlinear observations and...

Meng-Ge Zhou, Xiao-Qun Cao, Yan Chen et al. · 0 citations
Preprint Sep 2026

Toward Principled Generative Data Assimilation of Turbulent Flows from Sparse Observations

Turbulent flows are highly chaotic, which makes their instantaneous states difficult to predict. Data assimilation aims to reduce this predictive uncertainty by synthesizing the predictions of a physical solver with partial observations of the system. Traditional data assimilation methods such as ensemble and variation...

B. Turan, Zhuo-Ran Liu, Heng Xiao · 0 citations
#machine learning Preprint Sep 2026

Learning to Bias: Machine Learning-Enhanced Particle Filters

Neural Optimal Particle Filters are introduced, which learn an amortized approximation to the optimal proposal from offline simulated one-step conditioning tuples, and improve sample efficiency and distributional accuracy over standard PF baselines with modest computational overhead.

Apoorv Srivastava, Eric F. Darve · 0 citations
#machine learning Preprint Oct 2026

DAWIS: Data Assimilation with Windowed Inverse Sampling via Multitask Interpolants

Flow- and diffusion-based generative models have recently emerged as flexible and highly efficient forecasting models for dynamical systems. When combined with inference-time guidance, they offer a promising route to high-dimensional non-Gaussian data assimilation (DA), the problem of combining forecasts with observati...

Erik Wikingsson, Martin Andrae, Tomas Landelius et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.