Prompt Learning for Textual Heterogeneous Information Networks

Abstract

Heterogeneous information networks (HINs) play an indispensable role in a wide range of domain-specific applications, from recommender systems to conversational platforms. Textual HINs are graphs with abundant textual information. Currently, most advanced approaches to mine textual features from HINs follow a “pre-training and fine-tuning” schema, which may cause a “negative transfer” problem since there is a gap between pre-training tasks and downstream tasks. We propose a prompt-learning framework P-HIN that provides a new angle to align textual information and graph information, while narrowing down the gap between the pre-trained models and various downstream tasks. To the best of our knowledge, we are among the first to introduce and exploit the idea of prompt learning to align HIN features and textual features. The proposed framework P-HIN is composed of a text encoder and a graph encoder and uses contrastive learning to align and fuse the graph–text pair. This pre-training operation naturally fits the few-shot learning setting. For the graph encoder, we introduce two graph pre-training tasks, masked node modeling and edge reconstruction, to exploit self-supervised information. During optimization, instead of handcrafted prompts, we use a learnable continuous text that enables more efficient and task-relevant transfer to downstream datasets. We consider a residual connection to use the context from the graph to prompt the text encoder. In experiments, P-HIN consistently and significantly outperforms state-of-the-art alternatives on all real-life datasets.