Activation Function Effects in Physics-Informed Neural Networks for One-Dimensional Signal Prediction

Abstract

Physics-informed neural networks provide a mesh-free way of solving differential equations by embedding governing physical laws directly into the training objective. This paper evaluates their suitability for one-dimensional signal prediction governed by representative partial differential equations. Three equations with different physical behavior are considered: the heat equation, the wave equation, and the viscous Burgers equation. A common fully connected physics-informed architecture is implemented in DeepXDE and trained using a two-stage Adam and L-BFGS optimization procedure. The study compares three activation functions: hyperbolic tangent, sinusoidal activation, and swish. The results show that the activation function has a significant effect on convergence and approximation quality. Swish provides the lowest loss for the smooth diffusive heat equation, tanh is the most robust option for the nonlinear Burgers equation with steep gradients, and both tanh and sinusoidal activations are suitable for oscillatory wave behavior. The findings confirm that activation selection should reflect the expected structure of the modeled signal rather than be treated as a generic architectural choice.