Shaping Graph Neural Networks with Dynamical Systems
TL;DR
This work presents a unified dynamical-systems perspective for shaping approaches that explicitly control and regulate the degree of propagation, conservation, and dissipation of information throughout the neural flow, and highlights how neural differential equations provide a coherent theoretical framework for designing graph architectures with controllable stability, memory retention, and long-range information propagation guarantees.
Abstract
The dynamics of information diffusion in Graph Neural Networks (GNNs) is a key issue that heavily influences graph representation learning, especially when long-range propagation is required. In this work, we present a unified dynamical-systems perspective for shaping approaches that explicitly control and regulate the degree of propagation, conservation, and dissipation of information throughout the neural flow. By interpreting GNN layers as discretizations of continuous-time differential equations defined over graphs, we leverage tools from stability theory, Hamiltonian mechanics, and wave dynamics to design architectures with principled (long-range) propagation properties. We review and analyze three complementary formulations: antisymmetric parameterizations that enforce non-dissipative behavior via spectral control of the Jacobian, port-Hamiltonian and oscillatory dynamics that embed conservation laws directly into the architecture. Across long-range graph transfer and graph property prediction benchmarks, these differential-equation-inspired GNNs consistently outperform classical message-passing and transformer-based models, maintaining stable information flow even in extreme propagation regimes. More broadly, this work highlights how neural differential equations provide a coherent theoretical framework for designing graph architectures with controllable stability, memory retention, and long-range information propagation guarantees.
