Skip to content

IMoKGNN: Dual-Stream Fusion of Generic and Task-Specific Language Model Features for Graph Neural Networks

Sep 2026 · ACM Transactions on Intelligent Systems and Technology · 0 citations · 49 references
Advanced Graph Neural Networks

TL;DR

A dual-channel knowledge fusion framework, named MoKGNN, to simultaneously leverage the generic semantic knowledge from LLMs and the task adaptation capability of TLM, and a Knowledge Alignment Network to align the LLM embeddings with the TLM representations.

Abstract

Text-Attributed Graphs (TAGs) are prevalent in various real-world scenarios, where each node is associated with a text attribute. Representation learning on TAGs relies on a comprehensive understanding of both the textual attributes and the topological connections. Recent works have enhanced graph neural networks (GNNs) with pre-trained language models (PLMs) for textual attribute modeling, achieving promising results compared with shallow text representations such as BoW. With the advent of more powerful LLMs, how to effectively and efficiently inject the generic knowledge in LLMs into representation learning on TAGs still calls for systematic investigation. To this end, we first examine whether mainstream generative LLMs can provide high-quality generic text embeddings that can be directly used by downstream GNNs, and further explore their complementarity with the task-specific knowledge from a tuned PLM (TLM). Based on these observations, we propose a dual-channel knowledge fusion framework, named MoKGNN, to simultaneously leverage the generic semantic knowledge from LLMs and the task adaptation capability of TLM. Specifically, we design a dual-channel feature extraction module to learn generic and task-specific text representations separately. Next, we introduce a Knowledge Alignment Network to align the LLM embeddings with the TLM representations. Finally, MoKGNN adopts a global mixing weight to fuse the two channels, obtaining a unified node embedding for downstream GNN training. Moreover, we further analyze a key limitation of such early feature fusion under message passing: once heterogeneous text features are merged before neighborhood aggregation, source-specific noise and non-aligned semantics may be diffused together and become harder to disentangle and correct. Motivated by this observation, we propose IMoKGNN, a decoupled dual-stream framework that propagates LLM and TLM features in parallel with a shared GNN backbone and performs late fusion in logit space. Extensive experiments on five TAG datasets demonstrate the effectiveness of our approaches for both node classification and link prediction, and show that IMoKGNN yields more reliable improvements than MoKGNN.

View source

Similar papers

Book Open access Nov 2026

LLMTGL: Large Language Models for Robust Text-Attributed Graph Learning under Missing Text and Label Noise

Text-attributed graphs have attracted considerable attention due to their extensive applications in domains such as social networks and citation networks. Existing methods typically rely on reliable text-attributed graphs. However, the frequent occurrence of missing text in real-world scenarios severely constrains mode...

Riting Xia, Guang-Liang Zhao, An-Chen Li et al. · 0 citations
#graph neural networks Open access Oct 2026

GraphLLM4CTR: Large Language Model-enhanced Graph Neural Networks for Click-through Rate Prediction in Recommender Systems

Click-through rate (CTR) prediction is a crucial task to estimate the probability that users will click on items in online platforms. Recently, researchers have incorporated semantics derived from large language models (LLMs) into representations of users and items in a graph neural network (GNN) framework to facilitat...

Shan Gao, Yan-Wu Yang · 0 citations
Preprint Nov 2026

ReCoG: Relative Node Contexts for Multi-Domain Graph Pre-training

The success of foundation models in language and vision has motivated Graph Foundation Models (GFMs), which learn transferable knowledge through multi-domain graph pre-training. Existing methods advance GFMs through feature unification and structural learning, and in doing so they rely on the raw graph structure to org...

Yu-Peng Tong, Lianze Shan, Ji-Tao Zhao et al. · 0 citations
Preprint Oct 2026

Uncovering Semantic Hierarchies in Text-Attributed Graphs via Variational EM-based LLM-GNN Synergy

While hierarchical structures are intrinsic to organizing human knowledge, current representation learning on text-attributed graphs predominantly operates on flat semantic spaces, overlooking the rich coarse-to-fine granularity inherent in real-world data. To bridge this gap, we propose SHiFT, which synergizes the rea...

Yunhui Liu, Xu-Dong Jin, Qizhuo Xie et al. · 0 citations
Book Open access Nov 2026

PRISM: Reliable LLM Knowledge Distillation from Graph-Context Perspectives for Text-Attributed Graph Learning

Text-attributed graph (TAG) learning aims to leverage both textual attributes and graph structures for node classification. Recent studies have utilized Large Language Models (LLMs) to generate pseudo-labels and rationales for few-label TAG learning. However, existing approaches typically rely solely on the textual att...

Jung-Hyun Kim, Sooho Jung, D. Y. Kang et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.