Preprint

Unsupervised Adaptation of PDE Foundation Models

Ziye Song Zhao Wei Xin Yu I. Tsang Yueming Lyu

Aug 2026 · 0 citations · 56 references

Computer Science

TL;DR

This work proposes an unsupervised PDE-based finetuning framework that eliminates the need for ground-truth solutions, and introduces NSLoRA, a Newton-Schulz orthogonalized variant that rebalances adaptation.

Abstract

Pretrained partial differential equation (PDE) foundation models can generalize across different equations, but adapting them to unseen PDE systems typically requires dense solution data, which is often expensive or unavailable. To address this limitation, we propose an unsupervised PDE-based finetuning framework that eliminates the need for ground-truth solutions. We first pretrain a neighborhood attention Transformer on diverse time-dependent PDEs spanning varying spatial scales, yielding transferable representations across heterogeneous equations. In the adaptation stage, we construct a physics-based objective using the PDE residual and boundary conditions, and finetune the model on unseen equations via low-rank adaptation (LoRA). To address the uneven learning across physical quantities in standard LoRA, we introduce NSLoRA, a Newton-Schulz orthogonalized variant that rebalances adaptation. Our method achieves performance comparable to supervised LoRA finetuning without requiring any ground-truth solutions, while consistently outperforming competitive neural operator baselines and recent PDE foundation models across heterogeneous PDE benchmarks spanning multiple spatial dimensions.

arXiv Semantic Scholar

View source

Similar papers

#machine learning Preprint Sep 2026

Equation Recast for Canonical Operator Learning Across Parametric PDEs

Equation recast is introduced, which reformulates parametric operator learning as the learning of a single canonical operator, and provides a route toward reusable neural PDE solvers combining equation-guided transfer, data efficiency, and monitorable inference.

Qi-Yun Cheng, Valentin Duruisseaux, C. Clauser et al. · 0 citations

Preprint Aug 2026

Distillation of Foundation Models for Time-dependent PDEs

This work proposes Teacher Rollout Extension (TREX), a knowledge distillation framework that transfers the predictive capability of a pretrained foundation model into a compact and efficient student that can match or surpass the teacher's accuracy while reducing the number of parameters by several orders of magnitude a...

Daniel Musekamp, Boshra Ariguib, Andrei Manolache et al. · 0 citations

#artificial intelligence Preprint Sep 2026

When Known Physics Helps Neural PDE Models: Residual Constraints Out-Regularize Generic Priors for Nonlinear Dynamics

Neural PDE surrogates increasingly incorporate structural priors, yet it is often unclear whether their gains arise from physics-specific information or simply from regularization and training choices. We evaluate several such priors under a common protocol against a matched from-scratch neural operator baseline. Our c...

Z. Farazpay, Aniruddha Bora · 0 citations

Preprint Aug 2026

STCO: Conditional Neural Operators for Time-Dependent PDEs

This work introduces the Spatiotemporal Conditional Operator (STCO) for prescribed-condition operator learning (PCOL), a common interface that supplies prescribed target-time condition fields to heterogeneous backbone architectures while retaining their architecture-specific core computation and context pathways.

Xing-Xin Yang, Zhan Zhang, Juan Li · 0 citations

#machine learning Preprint Sep 2026

Local gradient neural operator

This work proposes the local gradient neural operator (LGNO) as a lightweight and interpretable alternative for field temporal evolution prediction and source identification in typical mechanical problems and exhibits wide applicability to mechanical problems including diffusion, flow, and quantum phenomena.

Bai-Ming Zhang, Jin-Song Tang, Ying Xu et al. · 0 citations

Preprint Aug 2026

Physics-Informed Foresight Pruning for Sparse PINN Solvers of Nonlinear PDEs

Small-batch PINN-NTK diagnostics further show that residual fidelity, solution accuracy, and kernel conditioning are distinct objectives, motivating pruning methods that explicitly balance solution-side and residual-side training dynamics during optimization.

A. Karimi, Uvini Balasuriya Mudiyanselage, Kookjin Lee · 0 citations