This work proposes a fully probabilistic extension of the manifold meta-learning framework, based on amortized Variational Inference, where a generative prior over the low-dimensional parameter manifold is learned.
Abstract
Deep learning has proven highly effective for nonlinear system identification, but heavily parameterized neural networks are prone to overfitting in low-data regimes and lack reliable uncertainty quantification. The recently developed manifold meta-learning framework addresses the data efficiency problem by restricting the model parameters to a meta-learned low-dimensional manifold. However, that method is purely deterministic. We propose a fully probabilistic extension of the manifold meta-learning framework, based on amortized Variational Inference, where a generative prior over the low-dimensional parameter manifold is learned. During task-specific adaptation, we combine Maximum A Posteriori estimation with the Laplace approximation to yield a mathematically grounded posterior approximation. Evaluated on a static regression task and the Bouc--Wen dynamical system benchmark, the proposed approach achieves predictive accuracy comparable to its deterministic counterpart while successfully providing calibrated uncertainty bounds in severely low-data regimes.
A deep variational identification method for nonlinear state-space models based on a structured Gaussian posterior using a non-causal dilated residual convolutional network and a Markov structure with block-tridiagonal precision is proposed to ensure linear-time inference.
Kaiqi Fang, Guijun Ma, Yasen Wang et al.· International Journal of Net...· 0 citations
A physics-aware neural-network-based latent-space framework for reduced-order forward modeling and variational parameter estimation and the robustness of the proposed framework is demonstrated, which generally reduces calibration error and variability compared with the standard surrogate models.
Qiyao Zhou, Xujia Zhu, Pierre Joli et al.· 0 citations
It is suggested that copula-based feature representations combined with deep learning provide a flexible approach for heterogeneous treatment effect estimation, particularly in high-dimensional settings with complex covariate dependence.
Parameter identification in nonlinear dynamical systems is complicated by model-form uncertainty arising from systematic biases that violate the zero-mean error assumption of standard data assimilation methods. Recent neural-network-based approaches learn arbitrary bias corrections online. However, they require careful regularization to ensure unique solutions and carry computational overhead from ensemble propagation and in-situ training. We present a framework that integrates the parametrized-background data-weak formulation with attention-based parameter identification networks. It projects model error onto a dictionary of physically motivated spatial templates rather than learning arbitrary corrections. This provides uniqueness through hard subspace constraints rather than soft regularization penalties, albeit at the cost of restricting the representable bias space to patterns anticipated from domain knowledge. The framework is demonstrated on parameter estimation in the Rijke tube model, achieving robust generalization to out-of-distribution (OOD) bias patterns not seen during training. Ablation studies confirm the importance of state supervision for amplitude-sensitive parameters and temporal bias modeling for phase-sensitive parameters. Performance degrades gradually when bias patterns lie outside the template span. This indicates that the attention encoder extracts parameter information from bias-invariant features rather than relying critically on template-based bias capture. The physics-constrained and universal approximator approaches represent complementary points on the flexibility–efficiency tradeoff, suited to different operational contexts.
Y. Ju· Machine Learning: Science an...· 0 citations
The combination of traditional statistical models and neural network (NN) components into semi-structured hybrid models is an intriguing approach to construct models that, ideally, combine traditional interpretability with the unprecedented flexibility of NNs. In order to preserve interpretability, it is usually necessary to restrict the NN components to prevent them from dominating the model. However, existing methods that enforce structural constraints on their NN components severely limit their models'flexibility; in contrast, methods that only enforce weak, indirect constraints lose meaningful interpretability. The method we propose therefore leverages invertible residual neural networks (i-ResNets) to equip generalized linear models with both nonlinear parameter estimation and a flexible correction of their distributional assumptions while always retaining stochastic monotonicity of the modeled distribution in the (formerly linear) predictor. The i-ResNets correspond to a controlled deviation from identity and by constraining their Lipschitz constant one can rigorously limit and quantify how far the hybrid model deviates from its traditional counterpart. This enables a user-specifiable compromise between flexibility and interpretability without limiting the structure of nonlinear and interaction effects that can be learned. Furthermore, we develop specific inherent interpretation techniques for our model and enforce model identifiability through an adapted post-hoc orthogonalization.
Tom A. Splittgerber, Niklas Koenen, Marvin N. Wright et al.· 0 citations
A novel Koopman operator‐based framework that integrates deep neural networks with subspace identification that achieves high modeling accuracy with significantly reduced errors is proposed, offering an effective and robust solution for nonlinear system modeling.
Zixiang Yuan, Jie Ding, Dezhi Shen et al.· International Journal of Rob...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
