Skip to content
Conference Open access

Direct Projection of Visual Features into Linguistic Embeddings for Domain-Adapted Multimodal Learning

2026 · Proceedings of the ... International Conference Computational Linguistics in Bulgaria · 0 citations

Abstract

This paper introduces a novel multimodal learning framework that directly projects visual features into linguistic embeddings, enabling robust, domain-adapted object recognition across diverse categories. Leveraging state-of-the-art visual (Detectron2 Mask R-CNN) and language (TinyLlama) models, the architecture incorporates a trainable projection layer that maps convolutional visual features into the embedding space of a large language model. This approach addresses the challenge of bridging high-dimensional, region-based visual representations with sequential, token-based language embeddings using minimal fine-tuning. The model is evaluated on the specialised Multilingual Image Corpus (MIC21), a rich dataset containing over 21,000 images and more than 700 categories, extending beyond standard benchmarks and annotated with multilingual captions. Experimental results demonstrate the effectiveness of this system for multimodal tasks, including image captioning and visual question answering. The design supports scalable adaptation to new domains and tasks, offering a flexible solution for unified visual–linguistic representation learning.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.