Study of the efficacy of GNN layers in a slew of regression contexts from rank ordering, error minimization and insight extraction shows that deep convolutional GNNs, particularly GEN, are more effective at these tasks than attention-based GNNs, while other classical, theoretically-inspired GNNs remain competitive and efficient.
Abstract
Graph Neural Networks (GNN) facilitate effective prediction on graph data such as molecules, media networks and neural network blueprints. GNNs facilitate prediction through message passing techniques which define how information flows from a node to its neighbors. Due to the ubiquity of the graph data type, the development of newer and better GNNs has garnered much interest in the machine learning community. However, GNN evaluation and benchmarking is primarily driven by classification tasks. Thus, prospective GNN message passing layers are evaluated on their ability to outperform prior work in classification contexts. In contrast, GNNs are equally capable of performing scalar regression prediction, yet this class of problem is often overlooked when proposing new GNNs while the best classification GNNs are utilized in an a priori or off-the-shelf manner for regression problems. In response, this paper studies the efficacy of GNN layers in a slew of regression contexts from rank ordering, error minimization and insight extraction. Results show that deep convolutional GNNs, particularly GEN, are more effective at these tasks than attention-based GNNs, while other classical, theoretically-inspired GNNs remain competitive and efficient.
This work proposes RTA, a simple MLP-based framework that replaces structural message passing with label-aware retrieval and propagation and provides theoretical insights that connect retrieval-based aggregation to softmax-attention message passing and establish the robustness of retrieved-context supervision to mis-retrieved outliers.
Jintang Li, Yuhong Chen, Ruo-Fan Wu et al.· 0 citations
This work proposes a novel Adaptive Dual-level Collaborative GNN associated with an adaptive dual-level collaborative mechanism, and shows that the ADC-GNN can inject the learned high-level information back into the node level, forming a closed-loop, bidirectional optimization process.
DeltaGNN is introduced, to the best of the authors' knowledge, among the first scalable (featuring linear computational and memory complexity overhead) and generalizable (capable of effectively handling graphs with diverse homophily, density, and topology) architectures for long-range and short-range interaction detection.
Kevin Mancini, Islem Rekik· IEEE Transactions on Pattern...· 2 citations
This work addresses the first systematic benchmark of the sheaf design space through evaluating three diffusion mechanisms (neural sheaf diffusion, sheaf attention, and sheaf attention with Graph Attention Network v2), three restriction-map parameterizations, three stalk dimensions, and six modern GNN architectural components, within a message-passing reformulation that never assembles the heavy sheaf Laplacian.
Stefano Fiorini, Edoardo Coppola, Pietro Liò· 0 citations
This survey provides a comprehensive overview of OOD detection methods, organizing them into traditional and deep learning families before examining graph-aware GNN-based approaches and synthesizing the conceptual relationships between these paradigms through three working hypotheses: graph-aware transfer, structure-sensitive scoring, and hybrid objective design.
Mao Nguyen, Thien Pham, Long Song Thien Nguyen et al.· IEEE Access· 0 citations
Graph neural networks (GNNs) have become a leading paradigm for graph learning, powering numerous applications, such as recommender systems in e-commerce and social platforms. However, GNN inference requires expensive propagation and aggregation of node features and feature gradients, repeated across layers, the cost of which grows with the graph size. To reduce the inference cost, recent GNN-to-MLP distillation methods use vector quantization (VQ) to discretize node embeddings for MLP prediction, which introduce additional parameters such as codebook size during pre-training that need to be manually tuned to avoid accuracy drop, and which require an extra MLP training stage. In this paper, we propose NodeBits, a lightweight plug-and-play framework that accelerates graph inference via a novel post-hoc binary quantization scheme. Unlike existing GNN-to-MLP distillation methods, NodeBits decouples quantization from GNN training, so can be directly applied to any trained GNN models to accelerate their inference % with negligible accuracy drop. while maintaining competitive predictive performance. The learned node representations are discretized into compact bit strings via a data-aware binary quantizer called DaBQ, which leverages lightweight data statistics on node representations to minimize the information loss induced by binarization. During inference, NodeBits reconstructs node embeddings from these bit strings and feeds them directly to the pre-trained readout layer, avoiding training an extra MLP. To further offset information loss of binarization, we introduce NodeBits^*, which fine-tunes the readout layer to better preserve model accuracy. Extensive experiments on 20 benchmark datasets demonstrate that NodeBits achieves an accuracy comparable to full-precision GNN inference while being orders of magnitude faster. NodeBits consistently outperforms existing GNN-to-MLP methods in both accuracy and inference speed.
Qihao Cheng, Tianhao Wu, D. Yan et al.· Proceedings of the 32nd ACM...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
