Sep 2026· ACM Transactions on Intelligent Systems and Technology· 0 citations· 49 references
Advanced Graph Neural Networks
TL;DR
A dual-channel knowledge fusion framework, named MoKGNN, to simultaneously leverage the generic semantic knowledge from LLMs and the task adaptation capability of TLM, and a Knowledge Alignment Network to align the LLM embeddings with the TLM representations.
Abstract
Text-Attributed Graphs (TAGs) are prevalent in various real-world scenarios, where each node is associated with a text attribute. Representation learning on TAGs relies on a comprehensive understanding of both the textual attributes and the topological connections. Recent works have enhanced graph neural networks (GNNs) with pre-trained language models (PLMs) for textual attribute modeling, achieving promising results compared with shallow text representations such as BoW. With the advent of more powerful LLMs, how to effectively and efficiently inject the generic knowledge in LLMs into representation learning on TAGs still calls for systematic investigation. To this end, we first examine whether mainstream generative LLMs can provide high-quality generic text embeddings that can be directly used by downstream GNNs, and further explore their complementarity with the task-specific knowledge from a tuned PLM (TLM). Based on these observations, we propose a dual-channel knowledge fusion framework, named MoKGNN, to simultaneously leverage the generic semantic knowledge from LLMs and the task adaptation capability of TLM. Specifically, we design a dual-channel feature extraction module to learn generic and task-specific text representations separately. Next, we introduce a Knowledge Alignment Network to align the LLM embeddings with the TLM representations. Finally, MoKGNN adopts a global mixing weight to fuse the two channels, obtaining a unified node embedding for downstream GNN training. Moreover, we further analyze a key limitation of such early feature fusion under message passing: once heterogeneous text features are merged before neighborhood aggregation, source-specific noise and non-aligned semantics may be diffused together and become harder to disentangle and correct. Motivated by this observation, we propose IMoKGNN, a decoupled dual-stream framework that propagates LLM and TLM features in parallel with a shared GNN backbone and performs late fusion in logit space. Extensive experiments on five TAG datasets demonstrate the effectiveness of our approaches for both node classification and link prediction, and show that IMoKGNN yields more reliable improvements than MoKGNN.
Text-attributed graphs have attracted considerable attention due to their extensive applications in domains such as social networks and citation networks. Existing methods typically rely on reliable text-attributed graphs. However, the frequent occurrence of missing text in real-world scenarios severely constrains mode...
Riting Xia, Guang-Liang Zhao, An-Chen Li et al.· Proceedings of the 35th ACM...· 0 citations
Click-through rate (CTR) prediction is a crucial task to estimate the probability that users will click on items in online platforms. Recently, researchers have incorporated semantics derived from large language models (LLMs) into representations of users and items in a graph neural network (GNN) framework to facilitat...
Shan Gao, Yan-Wu Yang· ACM Transactions on Knowledg...· 0 citations
The success of foundation models in language and vision has motivated Graph Foundation Models (GFMs), which learn transferable knowledge through multi-domain graph pre-training. Existing methods advance GFMs through feature unification and structural learning, and in doing so they rely on the raw graph structure to org...
Yu-Peng Tong, Lianze Shan, Ji-Tao Zhao et al.· 0 citations
While hierarchical structures are intrinsic to organizing human knowledge, current representation learning on text-attributed graphs predominantly operates on flat semantic spaces, overlooking the rich coarse-to-fine granularity inherent in real-world data. To bridge this gap, we propose SHiFT, which synergizes the rea...
Yunhui Liu, Xu-Dong Jin, Qizhuo Xie et al.· 0 citations
Text-attributed graph (TAG) learning aims to leverage both textual attributes and graph structures for node classification. Recent studies have utilized Large Language Models (LLMs) to generate pseudo-labels and rationales for few-label TAG learning. However, existing approaches typically rely solely on the textual att...
Jung-Hyun Kim, Sooho Jung, D. Y. Kang et al.· Proceedings of the 35th ACM...· 0 citations
Machine-learning networks can learn without detailed instructions from a designer. Those that do may offer insight into how the brain reorganizes itself during learning.
MIT News · Artificial Intelligence· news.mit.eduJul 15, 2026
Assistant Professor Pat Pataranutaporn describes a new interface that lets everyday users glimpse inside an AI's neural network before their chatbot ever says a word.
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.