Skip to content

Oracle, will I ever learn? A study of prediction convergence and complementarity across link prediction models

Sep 2026 · 0 citations · 36 references
Computer Science

TL;DR

This work proposes to measure model complementarity through the performance of an oracle that, for each query, selects the best prediction among a considered set of models, hence providing an upper bound on the performance achievable through model combination.

Abstract

Knowledge graphs have become an important source of structured knowledge for Web applications, including search, question answering, and recommender systems. In these applications, link prediction can serve either as a prediction task itself or as a means to enrich incomplete knowledge graphs for downstream tasks. Interestingly, different link prediction models, or even different training runs of the same model, can produce substantially different predictions for the same query. This suggests a variability in the capture of the underlying knowledge by models, thus raising a fundamental question: to what extent do different models capture complementary knowledge, and how much of this knowledge could be recovered by combining them? We propose to measure model complementarity through the performance of an oracle that, for each query, selects the best prediction among a considered set of models, hence providing an upper bound on the performance achievable through model combination. Across several architectures and benchmarks, we find a substantial gap between individual models and their oracle, revealing that different models capture complementary knowledge. Yet, this complementarity rapidly saturates as more models are added, leaving a persistent subset of queries unsolved even by a large number of models. These findings reveal both the potential of model complementarity and a fundamental limit to what current link prediction models can collectively recover; thereby highlighting the need for further research to build robust Web applications.

View source

Similar papers

Book Open access Sep 2026

From Side Information and Knowledge Graphs to Large Language Models: Two Decades of Knowledge Integration in Recommender Systems

Over the past two decades, Recommender Systems (RSs) have repeatedly changed how they integrate external knowledge, moving from constraints and side information to Knowledge Graphs (KG) and, more recently, Large Language Models (LLMs). This Past/Present/Future retrospective interprets that evolution as a sequence of re...

Claudio Pomo, Diego Baquero Sanz, Liam Claude Morris et al. · 0 citations
#artificial intelligence Preprint Sep 2026

ImbalancE: Inference-Time Latent Search Against Degree Imbalance in Link Prediction

Knowledge Graph Embedding models have been extensively used to learn representations of entities and relations in Knowledge Graphs for predicting missing links. However, the quality of the learned representations varies a lot across different areas of the graph. If previous research has loosely linked the problem to re...

Alberto Bernardi, Luca Costabello, Christophe Guéret · 0 citations
#artificial intelligence Preprint Sep 2026

From Parameters to Answers: How LLMs Retrieve and Use Their Internal Knowledge

This work compares country-continent questions with noun, adjective, and code answers while keeping several fitted measurements distinct across Qwen, Llama, and Gemma to separate early readability, natural strength, causal steering, and later content dependence.

Wen-Lin Wei, Yuan Fang, Ren-He Jiang et al. · 0 citations
#machine learning Preprint Aug 2026

Neural Regression with Embeddings for Numerical Attribute Prediction in Knowledge Graphs

A neural regression model (LitEm) that enables transductive knowledge graph embedding models to predict numerical attributes within knowledge graphs and a co-training framework that jointly trains state-of-the-art transductive knowledge graph embedding models with LitEm, which improves link prediction performance mainl...

Rupesh Sapkota, Louis Mozart Kamdem Teyou, Moshood Yekini et al. · 0 citations
Sep 2026

EvoSkillRec: Skill-Genome Evolution for Recommender Architecture Discovery

This work introduces EvoSkillRec, a promotion-and-reuse framework for cumulative recommender architecture evolution that decomposes recommenders into atomic executable skills and represents architectures as typed skill genomes, with each skill equipped with input--output types, semantic annotations, and implementation...

Xiao-Peng Li, Kuo Cai, Bo Chen et al. · 1 citation
Open access Sep 2026

To Learn or Not to Learn? An In-depth Empirical Analysis of Community Search Approaches

This paper systematically evaluates the overall performance of representative CS approaches, investigates the impact of intrinsic properties of ground-truth communities, and analyzes the approaches' generalizability within a unified experimental framework to advocate generalizability as a pivotal direction for future r...

Yu-Han Zhou, Zheng Wu, Qing Liu et al. · 0 citations

Related blog posts

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.