HGMRP extends single-relation prompts into a composite structure that includes both relation-specific and shared components, thereby enhancing expressive capability and significantly out-performs existing heterogeneous prompt methods, validating its effectiveness and superiority.
Graph prompt learning is an effective paradigm to adapt pre-trained graph models to downstream tasks in low-resource scenarios. However, existing multi-task graph pre-training frameworks generally use randomly initialized prompts, leading to poor alignment between the prompt space, pretext objectives and graph structural characteristics. This greatly weakens the task relevance, structural awareness and transferability of prompt representations. To address this challenge, we propose TPGC, a dual-prior prompt initialization solution that explicitly models the synergy between task prior and structural prior. Specifically, the Task-Prior Injection Module first conducts a short homologous multi-task pre-training on an auxiliary graph, enabling prompt initialization to inherit optimization preferences associated with multiple pretext tasks. Built on the task-aware representations, the Structure-Prior Injection Module further extracts transferable global structural context from the auxiliary graph, converting it into layer-wise prompt vectors by aggregating structurally informative node embeddings. Extensive experiments on 6 mainstream benchmarks covering node and graph classification show that TPGC achieves consistently better performance under few-shot settings than state-of-the-art baselines, with fewer downstream tunable parameters and lower runtime. The code is available at https://github.com/Virgilqiu/TPGC
Zhixuan Qiu, Yangtao Wang, Xiaocui Li et al.· 0 citations
Heterogeneous information networks (HINs) play an indispensable role in a wide range of domain-specific applications, from recommender systems to conversational platforms. Textual heterogeneous information networks (HINs) are graphs with abundant textual information. Currently, most advanced approaches to mine textual features from HINs follow a “pre-training and fine-tuning” schema which may cause a “negative transfer” problem since there is a gap between pre-training tasks and downstream tasks. We propose a prompt-learning framework P-HIN that provides a new angle to align textual information and graph information, while narrowing down the gap between the pre-trained models and various downstream tasks. To the best of our knowledge, we are among the first to introduce and exploit the idea of prompt learning to align HIN features and textual features. The proposed framework P-HIN is composed of a text encoder and a graph encoder, and uses contrastive learning to align and fuse the graph-text pair. This pre-training operation naturally fits the few-shot learning setting. For the graph encoder, we introduce two graph pre-training tasks, masked node modeling and edge reconstruction, to exploit self-supervised information. During optimization, instead of handcrafted prompts, we use a learnable continuous text that enables more efficient and task-relevant transfer to downstream datasets. We consider a residual connection to use the context from the graph to prompt the text encoder. In experiments, P-HIN consistently and significantly outperforms state-of-the-art alternatives on all real-life datasets.
Yang Fang, Xiang Zhao, Daojian Zeng et al.· ACM Transactions on Informat...· 0 citations
Click-through rate (CTR) prediction is a critical task in personalized recommender systems. Existing methods that align collaborative information from conventional CTR models with semantic information from pre-trained language models (PLMs) have demonstrated superior performance compared to approaches relying on a single information source. However, most of them perform alignment at the embedding level, which introduces noise from heterogeneous vector spaces and limits fine-grained semantic mapping. Moreover, these models highly depend on tabular features, thereby limiting their transferability. To address these challenges, we propose to conduct Multi-scale Graph Tokens Alignment (MGTA) for CTR prediction via pre-trained language models, which enables deep cross-modal information alignment while maintaining strong generalizability. Specifically, MGTA first captures multi-scale graph tokens rich in collaborative signals by decoupling and quantizing graph structures based on graph neural networks (GNNs), and then achieves token-level alignment between collaborative signals and semantic knowledge via PLM fine-tuning. To achieve efficient transfer with MGTA, we further introduce the Cross-domain Token Adapter that enables collaborative signals adaptation by mapping graph tokens from the target domain to the source domain, which necessitates only the injection of target-domain semantic knowledge, in turn reducing fine-tuning time. Extensive experiments on three real-world datasets demonstrate the effectiveness of MGTA compared to existing baselines.
Zhongzheng Wu, Yating Ren, Shuocheng Li et al.· Proceedings of the 32nd ACM...· 0 citations
A novel transferable graph prompt attack, called TGPA, is proposed, which shifts the attack paradigm by introducing a hierarchical structural decoupling mechanism, which reduces the performance of pre-trained graph models with graph prompts by up to 28.9%, while guaranteeing robustness, stealthiness, and transferability.
Ju Jia, Haonan Wang, Tian Wu et al.· Neural Networks· 0 citations
The proliferation of dynamic graph-structured data necessitates Continual Graph Learning (CGL) to enable models to learn incrementally and retain past knowledge. Class-incremental learning (class-IL) in CGL is particularly challenging due to catastrophic forgetting. Existing CGL strategies, including recent prompt-based learning methods, often grapple with significant memory overhead and suboptimal alignment between pre-training objectives and downstream continual tasks. We propose TriP, a triple-prompt framework aligning pre-training and class-incremental objectives in continual graph learning. As a lightweight parameter isolation-based method, TriP utilizes feature-level and class-level prompts to precisely capture task-specific knowledge. It introduces a unified prompt template that aligns downstream classification with self-supervised pre-training objectives, which can bridge the semantic gap and maximize the generalization capabilities of pre-trained graph models. Extensive experiments on four public benchmark datasets demonstrate that our method significantly outperforms state-of-the-art approaches, illustrating its effectiveness in mitigating catastrophic forgetting and improving the efficiency of continual learning on graphs.
Can-Ming Cui, Hui-Yu Zhou, Pei-Yuan Lai et al.· Neural Networks· 0 citations
A combined HGT-based framework incorporating contrastive representation learning and deep clustering with multi-round training via pseudo-labels is presented, enabling the model to better deal with label scarcity, heterogeneous dependencies, and overlapping semantics in practical, complex, attributed networks.
Hamza Haddad, Hicham Attariuas, A. Younes· Edelweiss Applied Science an...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
