The ensemble generative filter (EnGF) is proposed, a simple yet effective method for non-Gaussian filtering that delivers clear and often substantial improvements over the EnKF, in some cases even approaching the filtering accuracy of a massive-ensemble particle filter at a small fraction of its cost.
Abstract
Sequential data assimilation (DA) faces a fundamental trade-off: particle filters capture non-Gaussian cycling priors but require prohibitively large ensembles, whereas the ensemble Kalman filter (EnKF) is computationally efficient but constrained by its Gaussian assumption. As machine learning enables rapid model forecasts, exploiting non-Gaussian prior features via moderately large ensembles has become increasingly viable. To exploit this opportunity, we propose the ensemble generative filter (EnGF), a simple yet effective method for non-Gaussian filtering. The key idea is to fit a generative model to the forecast ensemble at each DA cycle and harness its defining strength, inexpensive sampling, to draw a much larger particle population for Bayesian analysis without any additional model forecasts; we adopt a Gaussian mixture model as a lightweight instance that can be fit cheaply from a moderate ensemble. To address practical challenges, we further extend the EnGF by introducing (i) a tempered EnGF using likelihood tempering to prevent particle degeneracy under informative observations and (ii) a latent EnGF that performs prior modeling and Bayesian updates in a reduced latent space for high-dimensional systems. Across chaotic systems (doubling map, Lorenz-63, and Lorenz-96) and a challenging shock-tube problem, the EnGF delivers clear and often substantial improvements over the EnKF, in some cases even approaching the filtering accuracy of a massive-ensemble particle filter at a small fraction of its cost.
Bayesian data assimilation combines model forecasts with noisy observations, but sampling high-dimensional, non-Gaussian posteriors remains challenging. We introduce an observation-interpolant framework that turns pretrained stochastic interpolant, flow matching, and diffusion models into posterior samplers without ret...
A training-free adaptive ensemble conditional score filter is proposed, which recasts noisy posterior score estimation as estimating the conditional mean of the system state given a noisy reverse particle and the observation, and establishes a bound relating conditional-mean estimation errors to reverse-sampling endpoi...
Yang-Wen Zhang, Shi-Wei Ni, Xiao-Ping Zhang et al.· 0 citations
Particle filters provide a nonlinear Bayesian framework for data assimilation, but global weights often degenerate in high-dimensional geophysical systems. Ensemble Kalman filters are stable with practical ensemble sizes, yet their Gaussian and locally linear updates can be restrictive for nonlinear observations and...
Meng-Ge Zhou, Xiao-Qun Cao, Yan Chen et al.· Monthly Weather Review· 0 citations
Turbulent flows are highly chaotic, which makes their instantaneous states difficult to predict. Data assimilation aims to reduce this predictive uncertainty by synthesizing the predictions of a physical solver with partial observations of the system. Traditional data assimilation methods such as ensemble and variation...
Neural Optimal Particle Filters are introduced, which learn an amortized approximation to the optimal proposal from offline simulated one-step conditioning tuples, and improve sample efficiency and distributional accuracy over standard PF baselines with modest computational overhead.
Flow- and diffusion-based generative models have recently emerged as flexible and highly efficient forecasting models for dynamical systems. When combined with inference-time guidance, they offer a promising route to high-dimensional non-Gaussian data assimilation (DA), the problem of combining forecasts with observati...
Erik Wikingsson, Martin Andrae, Tomas Landelius et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.