TL;DR
This work proposes a visual question answering model based on autonomous multi-modal knowledge learning, in which large-scale multi-modal knowledge is first accumulated by constructing pseudo data, and then dynamically filtered and iteratively updated through an autonomous updating strategy to reduce noise and improve knowledge quality.
Dynamic Multi-Path Retrieval for KB-VQA (DMRAG) is proposed, which re-trieves candidates through multiple retrieval paths that capture complementary visual and semantic cues and performs Question-Adaptive Gated Fusion to balance contributions from different modalities according to the query’s information need.
Knowledge-based Visual Question Answering aims to answer questions about an image by integrating external knowledge with visual and textual information. Recent approaches often rely on in-context learning to prompt Large Language Models (LLMs) with multimodal context in a zero-shot or few-shot manner. However, we obser...
EKS is a novel framework that leverages entity relations in commonsense knowledge graphs to dynamically generate knowledge sentences relevant to both visual and textual entities and formulates knowledge selection as a relevance scoring problem, where semantic similarity is used to measure the relevance between knowledg...
Text-based visual question answering (TextVQA) requires reasoning over images containing rich textual content, often involving knowledge beyond what is directly observable. Existing methods fuse visual objects and OCR tokens but struggle when questions require external knowledge. Moreover, naively incorporating retriev...
Existing class-incremental learning methods struggle in multi-label scenarios (MLCIL) due to the inherent contradiction of learning objectives arising from co-occurring and incomplete labels. We argue that the core obstacle is the model's ambiguous boundary between known and unknown knowledge, which undermines historic...
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
