This study introduces a dual-branch, spectrally-gated architecture (DBSG-PINN) that splits low- and high-frequency components into separate subnetworks joined by an adaptive gate, and uses it to run a partially controlled ablation of frequency decomposition and spectral routing.
Abstract
Partial differential equations (PDEs) often have high-frequency and multi-scale features that neural networks struggle to approximate. Physics-Informed Neural Networks (PINNs) build the governing equations directly into training, but suffer from spectral bias: they learn low-frequency components faster than high-frequency ones. Techniques such as Fourier feature embeddings and sinusoidal activations address this, but most studies assume they help across the board without checking which spectral regimes actually benefit. We introduce a dual-branch, spectrally-gated architecture (DBSG-PINN) that splits low- and high-frequency components into separate subnetworks joined by an adaptive gate, and use it to run a partially controlled ablation of frequency decomposition and spectral routing. We test this on five one-dimensional benchmark PDEs, ranging from smooth, single-scale problems to oscillatory, multi-scale ones. Frequency decomposition helps most on the spectrally complex benchmarks, cutting relative $L_2$ error by up to $59.2\%$ on a multimodal wave problem, but gives little benefit on smoother PDEs. On one benchmark (1D Wave), it performs substantially worse than a simpler fixed-combination variant. The gate's benefit scales with how spectrally rich the target solution is: the full model's advantage over the ablations is largest on multi-scale benchmarks and smallest (or negative) on single-scale ones, consistent with the gate exploiting frequency structure rather than acting as noise,though we do not directly visualize or quantify its spatial activations in this study. All results come from a single training seed across five 1D benchmarks, so we present this as an exploratory study meant to raise questions rather than answer them, and outline the additional seeds and benchmarks needed to test whether the pattern holds.
This work proposes the Fourier-enhanced alternating Levenberg--Marquardt PINN (FALM-PINN), an optimization framework that decouples representation learning from coefficient fitting within a single nonconvex optimization objective.
Yulun Wu, Matthieu Barreau, Miguel Aguiar et al.· 0 citations
Deep neural operators learn mappings between input functions and complete PDE solution fields, enabling forward evaluations of new problem instances orders of magnitude faster than conventional numerical solvers. Attention mechanisms have recently been introduced into neural operators, but most studies change several architectural components at once, making it difficult to identify what actually improves accuracy. This work presents a controlled and systematic study of five deep operator network (DeepONet) variants with distinct attention mechanisms, trained under both data-driven and physics-informed regimes, to isolate the effects of cross-attention, self-attention, tokenization, and attention depth. We evaluate them on a source-driven transient one-dimensional nonlinear diffusion-reaction equation, a transient one-dimensional viscous Burgers equation with variable initial conditions, and a two-dimensional Poisson heat-conduction problem with heterogeneous source fields. Per-sensor tokenization with cross-attention reduces the mean relative L_2 error of the classical DeepONet in all benchmark-training combinations by factors of 2.4-28.0, while the best attention configurations reach 3.5-32.3. Branch self-attention paired only with dot-product fusion is inconsistent, degrading the one-dimensional problems while helping the more complex two-dimensional source field; added on top of cross-attention it improves all six cases, though by less than cross-attention fusion alone. Global pre-mixing provides no consistent benefit. Increasing cross-attention depth further improves accuracy, but with diminishing returns and a substantially higher cost under physics-informed training. Overall, query-dependent cross-attention is the most reliable mechanism, whereas branch self-attention is most useful for large, spatially complex functional inputs.
Amar Alem Koric, Qi-Bang Liu, S. Koric· 0 citations
Extensive numerical experiments demonstrate that PINNs-MSFF achieves superior accuracy, stability, and convergence, effectively capturing complex fractional dynamics, sharp localized gradients, and dispersive phase transitions where standard PINNs often fail.
Harender Kumar· International Journal of Dyn...· 0 citations
This work embeds feature interaction modules derived from factorization machines (FMs) into physics-informed neural networks (PINNs) and neural operator learning, to enhance model expressiveness for solution manifolds of parameterized partial differential equations (PDEs). Motivated by the second-order Taylor expansion of multivariate functions to characterize variable couplings, we first propose FM-PINN. It explicitly captures spatio-temporal variable interactions and improves the approximation accuracy for smooth high-order PDEs. We further group spatial coordinates, time, physical parameters, and initial and boundary conditions into independent feature sets and model their cross-group interactions. Based on this strategy, we develop FM-Operator and FM-DeepONet, which are particularly effective for nonlinear conservation laws and problems with sharp gradients or discontinuities, while offering no consistent advantage on smooth operator learning benchmarks. Numerical tests demonstrate that the proposed mechanism delivers substantial accuracy gains on challenging shock-dominated equations, indicating a promising direction for physics-consistent modeling of parameterized PDEs with strong cross-field dependencies.
Machine learning-based partial differential equations (PDEs) solvers have attracted significant attention in recent years. Most progress in this area has been driven by deep neural networks such as physics-informed neural networks (PINNs) and kernel method (such as physics-informed Gaussian Processes). We introduce a physics-informed random feature method for countering part of the spectral bias which PINN-based solvers are facing for a certain class of PDEs. Random feature method was originally proposed to approximate large-scale kernel machines and can be viewed as a specialized randomized neural network. Compared to other state-of-the-art PINN-based solvers which require a large number of collocation points, our proposed method reduces the computational complexity. In this paper, we develop a rigorous approximation error analysis and derive high-probability error bounds on the $H^1$ norm. We provide extensive numerical tests for verifying our theoretical guarantees on error decay rates, as well as several comparison tests to showcase our claimed capability for combating spectral bias in these deep learning based methods.
Chi-An Chen, Chun-Yang Liao, Ming Zhong· 0 citations
The growing application of physics-informed neural networks (PINNs) for solving parametric partial differential equations (PDEs) in fluid dynamics has demonstrated their potential for modeling complex multiscale flows; however, conventional PINNs often exhibit spectral bias and slow, unstable convergence, limiting accuracy in boundary layers and wakes. This research presents novel physics-informed feature decomposition in residual dense block neural networks (PI-RDB-NN), which embeds physical constraints directly into the network architecture rather than relying solely on soft constraints. PI-RDB-NN uses hierarchical residual dense blocks for multi-scale feature extraction, allocates feature channels to velocity and pressure in a 2:1 ratio consistent with two-dimensional incompressible Navier–Stokes physics, and enforces mass conservation via a learnable divergence-aware projection applied at the feature level. The model is evaluated on National Advisory Committee for Aeronautics (NACA) 0012 airfoil flow at Reynolds numbers (Re)=5000 and Re=1000 using a hybrid loss combining PDE residuals, boundary conditions, and sparse computational fluid dynamics (CFD) data. PI-RDB-NN reduces PDE residual and divergence error by 91.2% and 71.7% vs traditional PINNs (Re=5000) and by 85.5% and 85.4% vs a physics-informed Deep Operator Network (DeepONet) baseline (Re=1000). These physics consistency gains improve aerodynamic force predictions and CFD agreement, confirmed by velocity, wake, and pressure coefficient (Cp) distributions. Consistent accuracy across both Reynolds regimes supports the framework's generality, with three-dimensional and unsteady extensions identified as future work.
Sarmad Iftikhar, Ishfaq Ahmad, Diltaj Ali et al.· The Physics of Fluids· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
