Optimal Transport Dropout (OTD) is introduced, which offers a way to learn structured predictive uncertainty without explicit posterior inference or ensembles of independently trained predictors.
Abstract
Deterministic neural networks and neural operators provide point predictions with no intrinsic measure of reliability. Yet, predictive uncertainty may stem from irreducible outcome variability, finite data, or limitations of the chosen model class. Monte Carlo dropout offers a computationally convenient way to construct a predictive distribution through stochastic feature masking, without training multiple independent networks or explicitly inferring a posterior over model parameters. However, its perturbation law is largely prescribed a priori and typically factorised across latent coordinates. We introduce Optimal Transport Dropout (OTD), which instead learns the predictive mapping and the law of its latent perturbations jointly. Starting from a simple independent reference distribution, OTD transports latent perturbations through a learnable flow and propagates them through the predictive neural network, thereby inducing a structured predictive law. Training uses the strictly proper Energy Score, while a kinetic-action term geometrically regularises the transport. Synthetic benchmarks show that OTD captures multimodal predictive distributions, generates meaningful dispersion when the model is misspecified, and exhibits contracting dispersion as more training data or greater model capacity are provided. For a field-valued partial differential equation surrogate, predictive dispersion strongly aligns with the spatial pattern of prediction errors. On this task, compared with Monte Carlo dropout, OTD yields more accurate predictions and better-calibrated, substantially narrower intervals. On real-world regression benchmarks, it further shows competitive accuracy and better probabilistic predictions compared to established baselines. OTD therefore offers a way to learn structured predictive uncertainty without explicit posterior inference or ensembles of independently trained predictors.
Physics-informed Gaussian processes (GPs) provide a principled framework for scientific inference by expressing uncertainty over empirical measurements, boundary conditions, and governing equations. However, they struggle to scale to larger datasets and nonlinear information. Conversely, diffusion models excel at learn...
Tobias Weber, Carla Sagebiel, Philipp Hennig· 0 citations
Variational modeling is highlighted as a complementary approach to robust long-horizon neural PDE dynamics by introducing latent Markov dynamics, which combines spatially resolved latent fields, structured Gaussian perturbations, and a neural-operator transition.
Jun-Yi Liao, J. Guilleminot, Vahid Tarokh· 0 citations
This work presents a framework for learning continuous latent representations of admissible partial differential equations by embedding a scientific inductive bias directly into the training distribution, and shows that embedding a scientific inductive bias in the training distribution enables the learning of compact a...
James Crowley, Faez Ahmed, A. van Beek· 0 citations
Numerical tests on synthetic datasets evaluate the sparse GKDE, the stochastic generator, and the complete UQ-ANN framework, including out-of-sample predictions.
Scientific prediction under uncertainty requires more than a single future state: it requires the distribution of possible outcomes. We show that this distribution-valued evolution can be learned directly as a stochastic solution operator between arbitrary source and target times. We introduce Stochastic Latent Twins,...
Matthias Chung, Jack Michael Solomon, Deepanshu Verma· 0 citations
SLiSDE, a family of Neural SDE models built from structured linear stochastic layers, is introduced and it is proved well-posedness, a discretisation error bound, validity of the change of measure, and a universality result: the terminal laws of the gated stack are dense in the space of square-integrable laws.
Francesco Piatti, Andrea Iannucci, Thomas Cass· arXiv.org· 0 citations
Related blog posts
Microsoft Research Blog· microsoft.comNov 16, 2026
AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.
MIT News · Artificial Intelligence· news.mit.eduNov 13, 2026