A nonasymptotic risk bound is established that disentangles pretraining representation error from labeled-sample complexity, formally quantifying the benefit of large-scale unlabeled data for downstream knowledge prediction.
Abstract
Knowledge graph learning provides a powerful framework for representing and inferring structured knowledge, with broad practical applications. However, the scarcity of relation-specific labeled triples per entity hinders the training of expressive models, and the ad hoc design of scoring functions limits generalizability and lacks theoretical grounding. We address both issues with a theoretically grounded, end-to-end training framework that extends and subsumes existing methods. Our framework is a two-stage procedure: unsupervised pretraining over heterogeneous corpora followed by supervised learning with multiple relation types. We establish a nonasymptotic risk bound that disentangles pretraining representation error from labeled-sample complexity, formally quantifying the benefit of large-scale unlabeled data for downstream knowledge prediction. Synthetic experiments validate each theoretical component, and real-world experiments confirm the effectiveness of our approach on large-scale knowledge graph benchmarks.
Graph neural networks (GNNs) are widely used to represent complex interactions and relationships among entities. We investigate a multimodal model that combines two complementary ideas: a self-supervised method that enables a GNN encoder pretrained on one dataset to operate directly on another dataset with a different node-feature dimensionality, without rebuilding the model or realigning the data; and an alternating optimization method that updates a language-model module in an E-step and a GNN module in an M-step, rather than jointly training a large language model and a GNN end to end on a large graph. Despite expectations, the combined model did not sufficiently improve predictive performance. We identify six factors: (1) an external anchor in the E-step has a strength-safety trade-off: a weak anchor has little effect, whereas an overly strong anchor can damage the graph representation; (2) the knowledge of the E-step teacher is not injected directly into the GCN embedding Z; (3) the representation space constructed in the M-step is not optimized for the same objective as the E-step teacher space, resulting in a compromise representation for target classification; (4) GCN propagation averages a node's own textual information with information from its neighbors; (5) cosine alignment does not guarantee axes that are discriminative for classification, so stronger geometric alignment with the E-step text anchor need not sufficiently improve the target decision boundary or classification performance; and (6) the force that preserves the source-side self-supervised geometry in the M-step conflicts with the force that moves the representation toward the E-step teacher. We support these observations through a staged set of experiments that varies the influence of the E-step.
Fumiaki Kimino, Ryoma Sato Sokendai, National Institute of Informatics· 0 citations
This framework performs LLM knowledge elicitation to extract factual knowledge from the model’s internal representations and transforms sentence-level representations into entity-level representations and aligns them within a unified space.
Deyu Chen, Qiyuan Li, Jinguang Gu et al.· 0 citations
Knowledge graph completion is one of the core tasks in the field of knowledge graphs, which predicts missing links through inference of existing facts. With the advancement of deep learning technology, utilizing end-to-end deep learning models for knowledge graph completion has become a cutting-edge research direction. However, the performance of current knowledge graph completion models is still limited by text quality and incomplete structure. To address this issue, this paper proposes a method of using large models for data augmentation to improve the inference performance of the model. Specifically, we first introduced a pre-extractor model based on a hybrid architecture of rules and neural networks, which is used to identify long tail entities in the dataset and generate several candidate tail entities through relationships. Then, we use this data to have the Large Language Model infer the most factual triplet. Finally, we use the enhanced dataset for predictive inference. SAGE achieves better results on three standard KGC datasets. For instance, on the FB15K-237 dataset, compared to the SimKGC baseline model, SAGE improves Hits@1 by 1%, Hits@3 by 0.9%, and Hits@10 by 1.6%.
Er-Zhuo Xu· Poster Volume 0008 The 2026...· 0 citations
SelfGraphRAG, a framework that generates question-answer pairs directly from knowledge graph structure and uses them to train a query-conditioned graph retriever, suggests that knowledge graph structure can provide useful supervision for training graph retrievers when labeled data are unavailable.
LUCID, an LLM-guided, interpretable, training-free, and unsupervised community detection method, designed as a four-stage pipeline that achieves state-of-the-art performance and consistently outperforms leading unsupervised and semi-supervised baselines.
Aoting Zeng, Kai Wang, Jianwei Wang et al.· 0 citations
Inductive link prediction in knowledge graphs refers to the task of inferring known relations between entities unseen during training. Most existing approach-es are limited to predicting only known relations and struggle to generalize to un-seen relations, which restricts their utility in dynamic settings. To address this challenge, we propose a novel inductive link prediction approach named RGIILP. Specifically, we construct a relation graph from the source knowledge graph and design a neural network model that enables interactive feature propagation be-tween entities and relations. Furthermore, we introduce the mutual information maximization mechanism between global and local representations to capture the global structural information of the graph. Experiments on several benchmark da-tasets demonstrate that RGIILP outperforms existing state-of-the-art methods for inductive link prediction task.
Hong-Bo Liu· Poster Volume 0008 The 2026...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
