This work proposes RTA, a simple MLP-based framework that replaces structural message passing with label-aware retrieval and propagation and provides theoretical insights that connect retrieval-based aggregation to softmax-attention message passing and establish the robustness of retrieved-context supervision to mis-retrieved outliers.
Abstract
Graph neural networks (GNNs) are typically conceptualized as message-passing neural networks, yet it remains unclear why neighborhood aggregation reliably outperforms node-wise multilayer perceptrons (MLPs). Despite its empirical success, this paradigm can be computationally expensive and sensitive to imperfect graph structures. In this work, we present a retrieval-augmented view of GNNs: each layer makes predictions by applying an MLP to a node representation together with a permutation-invariant summary of retrieved graph context. Motivated by this perspective, we propose RTA, a simple MLP-based framework that replaces structural message passing with label-aware retrieval and propagation. We provide theoretical insights that (i) connect retrieval-based aggregation to softmax-attention message passing, and (ii) establish the robustness of retrieved-context supervision to mis-retrieved outliers. Experiments on multiple text-attributed graph benchmarks show that RTA matches or even outperforms strong GNN and graph LLM baselines while improving efficiency and robustness across diverse scenarios.
Study of the efficacy of GNN layers in a slew of regression contexts from rank ordering, error minimization and insight extraction shows that deep convolutional GNNs, particularly GEN, are more effective at these tasks than attention-based GNNs, while other classical, theoretically-inspired GNNs remain competitive and efficient.
Keith G. Mills, Aedan J. DeFrates, Joong Ho Kim· arXiv.org· 0 citations
This work proposes a novel Adaptive Dual-level Collaborative GNN associated with an adaptive dual-level collaborative mechanism, and shows that the ADC-GNN can inject the learned high-level information back into the node level, forming a closed-loop, bidirectional optimization process.
DeltaGNN is introduced, to the best of the authors' knowledge, among the first scalable (featuring linear computational and memory complexity overhead) and generalizable (capable of effectively handling graphs with diverse homophily, density, and topology) architectures for long-range and short-range interaction detection.
Kevin Mancini, Islem Rekik· IEEE Transactions on Pattern...· 2 citations
Graph neural networks (GNNs) are widely used to represent complex interactions and relationships among entities. We investigate a multimodal model that combines two complementary ideas: a self-supervised method that enables a GNN encoder pretrained on one dataset to operate directly on another dataset with a different node-feature dimensionality, without rebuilding the model or realigning the data; and an alternating optimization method that updates a language-model module in an E-step and a GNN module in an M-step, rather than jointly training a large language model and a GNN end to end on a large graph. Despite expectations, the combined model did not sufficiently improve predictive performance. We identify six factors: (1) an external anchor in the E-step has a strength-safety trade-off: a weak anchor has little effect, whereas an overly strong anchor can damage the graph representation; (2) the knowledge of the E-step teacher is not injected directly into the GCN embedding Z; (3) the representation space constructed in the M-step is not optimized for the same objective as the E-step teacher space, resulting in a compromise representation for target classification; (4) GCN propagation averages a node's own textual information with information from its neighbors; (5) cosine alignment does not guarantee axes that are discriminative for classification, so stronger geometric alignment with the E-step text anchor need not sufficiently improve the target decision boundary or classification performance; and (6) the force that preserves the source-side self-supervised geometry in the M-step conflicts with the force that moves the representation toward the E-step teacher. We support these observations through a staged set of experiments that varies the influence of the E-step.
Fumiaki Kimino, Ryoma Sato Sokendai, National Institute of Informatics· 0 citations
CoRe-GNN is proposed, which performs both propagations in parallel at each layer: a coarsened inter-cluster term capturing long-range structure, and a local intra-cluster term preserving per-node discriminability.
Antonin Joly, Nicolas Keriven, Aline Roumy· 0 citations
The message passing mechanism, which updates node representations by exchanging messages with their neighbors, plays a critical role in graph neural networks (GNNs) for capturing structural patterns. Since the single message passing mechanism lacks the flexibility to handle graphs with differing node feature quality, GNNs with a fixed message passing mechanism, e.g., GCN, GAT, and GraphSAGE, exhibit inconsistent performance across different graphs. To address this issue, we propose Mixture of Message Passing (MoMP). Code is made publicly available at https://github.com/Tinmomo/MoMP, a novel mechanism that incorporates the Mixture of Experts (MoE) paradigm directly into the message passing mechanism of GNNs, treating different message passing mechanisms as ''experts''. Specifically, MoMP employs a gating network to perform edge-level routing, adaptively selecting the most suitable expert to process the message for each edge. To ensure training stability and efficacy, we also introduce a partially-shared expert architecture, reframing the learning task for sparse experts into a residual from a globally-learned general mechanism. Extensive experiments on several homophilic graph benchmarks demonstrate that the proposed method not only outperforms strong baselines but also provides significant interpretability. To the best of our knowledge, our work is the first to fuse the MoE paradigm directly with the GNN message passing mechanism, paving the way for developing adaptive GNNs.
Zhaojun Luo, Jintang Li, Yuchang Zhu et al.· Proceedings of the 32nd ACM...· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduJul 15, 2026
Assistant Professor Pat Pataranutaporn describes a new interface that lets everyday users glimpse inside an AI's neural network before their chatbot ever says a word.
Microsoft Research Blog· microsoft.comJul 13, 2026
Cryptographic code supports vital protections in modern computing systems. Learn how a new method helps verify code as developers write it while preserving speed and adaptability as it gets implemented and evolves. The post Verifying Rust cryptography in SymCrypt, from standards to code appeared first on Microsoft Research.
MIT News · Artificial Intelligence· news.mit.eduJul 6, 2026
PhD student Rachel Sava, winner of the Envisioning the Future of Computing Prize, explores transformative improvements and dystopian risks of neural technology.
