A multimodal core-opinion extraction framework in which visual evidence serves as a contextual anchor for textual judgment is proposed, providing a case-level value signal for downstream STI screening.
Abstract
Recent advances in large language models (LLMs) have reshaped semantic analysis. Opinion Extraction (OE) for Science and Technology Intelligence (STI) requires concise core opinions from large information streams. Off-the-shelf models struggle to filter noise from these streams and show limited structured-output reliability in zero-shot multilingual and multi-modal settings. To address information overload and extraction defocus, this study proposes a multimodal core-opinion extraction framework in which visual evidence serves as a contextual anchor for textual judgment. Using VideoLLaMA2 (VL2) and VideoLLaMA2.1 (VL2.1) as the base models, we apply Quantized Low-Rank Adaptation (QLoRA) fine-tuning on a curated dataset of 2,194 multilingual and multimodal samples. Under the selected Image-Augmented setting, fine-tuned VL2.1 generates structured JSON core-opinion outputs, achieving 64.98% Precision, 42.15% Recall, 51.14% F1-score, and 74.00% sample-level accuracy. Relative to the zero-shot VL2.1 setting, it raises the F1-scores of Spanish and Russian from 4.83% and 0.45% to 46.05% and 51.93%, respectively. The framework further incorporates a Fuzzy Cumulative Prospect Theory-based post-extraction triage module for case-level value assessment, providing a case-level value signal for downstream STI screening.
Multimodal aspect-based sentiment analysis (MABSA) predicts the sentiment expressed toward a target aspect by jointly using textual and visual information, supporting fine-grained opinion analysis in product reviews, brand monitoring, and customer feedback. However, existing approaches remain sensitive to irrelevant visual regions, weak text–image alignment, and limited use of external knowledge. Motivated by these challenges, this study systematically evaluated two text-only large language models and four open-weight large vision-language models for aspect-level sentiment classification. The open-weight models were adapted using 4-bit quantized low-rank adaptation, while GPT-4o was assessed under zero-shot, one-shot, and five-shot in-context learning without parameter updates. Experiments were conducted on Twitter-2015, Twitter-2017, and the seven-domain MASAD dataset and evaluated using accuracy and macro-F1. Among the evaluated multimodal models, Qwen3-VL-8B-Instruct achieves the strongest performance, reaching 83.22% accuracy and 81.72% macro-F1 on Twitter-2015, 79.50% and 78.93% on Twitter-2017, and up to 99.84% and 99.83% in the Plant domain of MASAD. From a symmetry perspective, semantically aligned text–image–aspect inputs provide consistent cross-modal evidence, whereas shuffled images introduce asymmetric, symmetry-breaking information. The resulting performance degradation under shuffled-image ablation indicates that reliable aspect-level sentiment prediction depends on preserving cross-modal semantic correspondence. These findings demonstrate the effectiveness of parameter-efficient LVLM adaptation for MABSA.
Ismail Ifakir, E. Nfaoui, Abderrahim Zannou· Symmetry· 0 citations
Multimodal long documents are core carriers of professional knowledge, where critical evidence is sparsely distributed across paragraphs and modalities. This easily causes key information omission and cross-modal hallucinations in summarization by multimodal LLMs. These issues stem from attention drift in long-range dependency modeling and gaps in inter-modal alignment. To address this, we introduce MMLDSum-Bench, a high-quality benchmark for multimodal long-document summarization, covering multiple domains, context-length scales, and visual-textual modality distributions. We further propose MMLDSum-LLM, a reproducible two-stage training framework that combines supervised fine-tuning with visual-alignment weighted loss and keyword-aware weighted loss, followed by GRPO with a multi-objective reward (keyword coverage, image-text alignment, ROUGE, and length control). Extensive experiments on MMLDSum-Bench, comparing against leading closed-source and open-source multimodal models under a unified evaluation protocol - including LLM-as-a-judge scoring, atomic-claim precision/recall, image-text alignment (ITA), and ROUGE - demonstrate that our approach significantly improves key-information coverage and cross-modal consistency.
The multimodal large language models (MLLMs) have achieved breakthroughs in remote sensing applications because of their powerful cross-modal understanding, enabling effective alignment between linguistic descriptions and visual data. However, the existing MLLM-based approaches still suffer from challenges, such as inaccurate region localization, limited domain adaptability, and high computational costs. To overcome these limitations, we propose a training-free visual prompting MLLM called TFVP-MLLM, which is built on visual prompting guidance and a scoring feedback mechanism to achieve high-quality remote sensing image classification under low resource constraints. Our TFVP-MLLM is composed of the category proposal generator (CPG) and the score feedback refiner (SFR). The CPG is responsible for generating high-quality candidate categories by integrating the proposed geographic priors injection module (GPIM) and hybrid-granularity region attention loss (HGRAL). Meanwhile, SFR uses an iterative generate and score process to enhance the accuracy and reliability of the candidate categories. As two critical components of CPG, GPIM incorporates prior geographic knowledge to significantly improve the model’s domain adaptation, while HGRAL optimizes visual token representations with several frozen modules to reduce computational requirements. To validate the effectiveness of the proposed TFVP-MLLM, the comprehensive experiments are conducted on three benchmark datasets UCM, AID, and DIOR-RSVG. For the scene classification task, TFVP-MLLM achieves the state-of-the-art performance among training-free multimodal approaches with overall classification accuracies of 97.8% and 82.4% on the UCM and AID datasets, respectively. For the regional object classification task, TFVP-MLLM achieves remarkable scores of 92.4% semantic similarity (SS) and 88.6% semantic intersection over union (SIOU) on the DIOR-RSVG dataset, which surpasses the other training-free methods and even achieves comparable results against supervised approaches. In addition, we perform a series of ablation studies to confirm the effectiveness of each module. The source code is available at https://github.com/Camellia86/TFVP-MLLM
Yifei Xu, Zi-Xiang Ni, Pingping Wei et al.· IEEE Transactions on Geoscie...· 0 citations
Experimental evaluation on the PoliMemeDecode1 dataset shows that the attention-based fusion significantly outperforms unimodal baselines and standard concatenation methods, achieving a state-of-the-art Macro-F1 of approximately 0.94.
Musa Tur Farazi, Nufayer Jahan Reza· arXiv.org· 0 citations
MGSI first encodes audio and visual streams at short-, medium-, and long-range temporal scales, preserving both local variations and global affective trends, and applies polarity- and intensity-aware enhancement to better handle ambiguous and near-neutral samples.
Shanshan Lin, Yuesheng Wu, Chao Chen et al.· 0 citations
Large Language Models (LLMs) have achieved remarkable success across a wide range of tasks. However, fine-tuning LLMs for Gloss-Free Sign Language Translation (GFSLT) remains a challenge. In this paper, we investigate how to effectively adapt LLMs to the GFSLT task. We show that there are two key issues that need to be solved: (1) the inherent distributional gap between visual feature inputs and text feature inputs makes it difficult for LLMs to interpret visual inputs; and (2) existing approaches typically concatenate visual and textual features in an autoregressive framework, which leads to the model overemphasizing textual inputs and deprioritizing visual cues, as LLMs are pretrained predominantly on text-centric data. To address the first challenge, we propose a simple yet effective method named Filtered Pseudo-Gloss CTC Pretraining, which leverages filtered pseudo-gloss sequences generated from text sequences to supervise the training of the visual backbone. To tackle the second issue, we introduce a Visual-Prioritized Distillation training strategy. Specifically, we define a visual-only prediction path in which text inputs are masked, and the model is required to generate the target sequence relying solely on visual inputs. To guide this path, the outputs from the standard visual-textual prediction are then distilled into the visual-only prediction path, encouraging the model to prioritize visual features. Comprehensive experiments and qualitative analyses demonstrate the effectiveness of the proposed model. The proposed SignLlama achieves very competitive performance on multiple datasets for GFSLT tasks, without using any extra modalities or external sign language datasets for pretraining.
Shiwei Gan, Xiao Liu, Yafeng Yin et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
