Multi-Modal Implicit Knowledge Graph Construction

Abstract

This paper proposes a novel approach to constructing knowledge graphs from implicit, multi-modal data. Traditional knowledge graph construction heavily relies on manual annotation, a process that is both time-consuming and expensive. Our method leverages the rich information contained within user behavior data – encompassing text, images, and videos – to automatically infer hidden knowledge and relationships. We employ deep learning models to represent these diverse modalities within a shared embedding space. Subsequently, we utilize similarity measures or graph neural networks to identify correlations and construct the knowledge graph. This approach significantly reduces the reliance on labeled data and offers a scalable solution for knowledge graph creation, particularly in scenarios where explicit knowledge bases are unavailable or incomplete. The core claim is that analyzing multi-modal user behavior data can effectively build a knowledge graph representing implicit knowledge and relationships. The key mechanism involves mapping different data modalities into a shared vector space using deep learning, followed by relationship discovery through similarity or graph neural networks. This work contributes to a more efficient and automated approach to knowledge graph construction, paving the way for intelligent applications that can reason and understand complex information.