Oct 2026· Journal of construction engineering and management· 0 citations· 32 references
Abstract
Timely and accurate recognition of workplace hazards is critical for ensuring safety in dynamic and high-risk environments such as construction sites. However, existing video-based approaches often rely on extensive annotations and fully supervised training, and many are tailored to specific hazard types, which limits scalability to rare or diverse scenarios. This study investigates whether a data-efficient vision–language framework can support early-stage hazard recognition from short preincident video segments under limited supervision. The proposed method builds on a video-adapted contrastive language-image pre-training (CLIP)-based vision–language backbone and introduces a parameter-efficient adaptation strategy that combines learnable visual prompting with low-rank adaptation (LoRA) applied to the text encoder. Learnable visual prompts capture global, summary, and local spatiotemporal cues from short video clips, and LoRA enables lightweight semantic adaptation without fine-tuning the full backbone. This design preserves most pretrained parameters and introduces only a small number of additional trainable parameters, making it well suited to few-shot learning in data-scarce safety scenarios. To evaluate early recognition capability, this paper further examines performance under shorter temporal observation windows by reducing the amount of video available for prediction. Experiments on a curated real-world hazard video data set show that the proposed method substantially improved few-shot recognition compared with a prompt-free baseline across five hazard categories and 5-, 10-, and 15-shot supervision. Overall, the findings suggest that combining temporal visual prompting with parameter-efficient text adaptation offers a scalable engineering pathway toward assistive hazard recognition in data-scarce environments, with promising but preliminary performance.
The multimodal large language models (MLLMs) have achieved breakthroughs in remote sensing applications because of their powerful cross-modal understanding, enabling effective alignment between linguistic descriptions and visual data. However, the existing MLLM-based approaches still suffer from challenges, such as inaccurate region localization, limited domain adaptability, and high computational costs. To overcome these limitations, we propose a training-free visual prompting MLLM called TFVP-MLLM, which is built on visual prompting guidance and a scoring feedback mechanism to achieve high-quality remote sensing image classification under low resource constraints. Our TFVP-MLLM is composed of the category proposal generator (CPG) and the score feedback refiner (SFR). The CPG is responsible for generating high-quality candidate categories by integrating the proposed geographic priors injection module (GPIM) and hybrid-granularity region attention loss (HGRAL). Meanwhile, SFR uses an iterative generate and score process to enhance the accuracy and reliability of the candidate categories. As two critical components of CPG, GPIM incorporates prior geographic knowledge to significantly improve the model’s domain adaptation, while HGRAL optimizes visual token representations with several frozen modules to reduce computational requirements. To validate the effectiveness of the proposed TFVP-MLLM, the comprehensive experiments are conducted on three benchmark datasets UCM, AID, and DIOR-RSVG. For the scene classification task, TFVP-MLLM achieves the state-of-the-art performance among training-free multimodal approaches with overall classification accuracies of 97.8% and 82.4% on the UCM and AID datasets, respectively. For the regional object classification task, TFVP-MLLM achieves remarkable scores of 92.4% semantic similarity (SS) and 88.6% semantic intersection over union (SIOU) on the DIOR-RSVG dataset, which surpasses the other training-free methods and even achieves comparable results against supervised approaches. In addition, we perform a series of ablation studies to confirm the effectiveness of each module. The source code is available at https://github.com/Camellia86/TFVP-MLLM
Yifei Xu, Zixiang Ni, Pingping Wei et al.· IEEE Transactions on Geoscie...· 0 citations
Automatic recognition and classification of vehicle damages is an important research direction in modern computer vision and artificial intelligence, playing an increasingly significant role in industrial and practical applications. Traditional computer vision-based approaches can recognize and classify objects with high accuracy; however, achieving task-specific performance typically requires large amounts of annotated data, time-consuming training or fine-tuning, and extensive parameter optimization. This process is not only resource- and cost-intensive but also limits the rapid adaptability of the technology. The aim of this research is to investigate how effectively the latest Multimodal Large Language Models (MLLMs) can recognize types of vehicle damage in a zero-shot setting, i.e., without fine-tuning, and to evaluate how their performance can be further improved through prompt engineering and fewshot prompting. MLLMs have the advantage of being able to provide multiple forms of information from a single query and supplement their outputs with natural-language explanations. In contrast, traditional models are generally designed to perform only one predefined task. Therefore, within the framework of this project, the performance of a fine-tuned YOLO-based computer vision model is compared with that of MLLMs in vehicle damage classification. This comparison highlights a modern, data-efficient approach that achieves competitive performance through prompt engineering and in-context learning, eliminating the need for additional model training and opening new directions for automotive applications.
Márk Mitrenga, B. Kővári, Péter Gáspár· International Conference on...· 0 citations
The application of automated traffic surveillance systems has become increasingly critical for improving traffic management and reducing reliance on manual monitoring. Traditional video surveillance methods are time-consuming, resource-intensive, and prone to human error, with operators frequently missing incidents due to fatigue and environmental factors. This study introduces a generative AI-based traffic surveillance system leveraging large language models (LLMs) to enable timely and context-rich interpretation of traffic events. We developed a custom-annotated dataset of 604 videos, capturing crashes, congestion, lane closures, and diverse weather and lighting conditions from Missouri Department of Transportation cameras and online sources. This approach fine-tunes the Qwen2.5VL-Instruct model using Low-Rank Adaptation (LoRA), temporal context enhancement, and multidimensional Rotary Position Embedding (mRoPE) for improved cross-modal fusion. Compared with the LLaVA-NeXT-Video baseline, the fine-tuned model achieves substantial gains in captioning performance (BLEU-4 = 0.4569, METEOR = 0.6187, CIDEr = 5.4612) and reduces average video description time from 35 seconds (manual) to 20 seconds (automated). An interactive web interface integrates real-time traffic monitoring with automatic incident detection and visualization, supporting faster emergency response and scalable deployment across traffic management centers. These results demonstrate the system's effectiveness in producing context-aware traffic scene descriptions, improving operational decision-making, and enhancing roadway safety.
Children's safety is the primary and core issue that kindergartens face. Traditional video surveillance is the main way for kindergartens to ensure safety, but this method highly relies on manual monitoring. It is not only inefficient but also prone to overlooking risks due to human negligence, and it is even more difficult to achieve real-time risk early warning. To solve these practical problems, we propose a real-time intelligent monitoring, safety early warning, and analysis framework based on a multi-modal large model. This framework integrates computer vision technology and large vision-language models, enabling both multi-dimensional scene perception and intelligent safety early warning and scene analysis. Specifically, for the needs of personnel identification and tracking, we fine-tuned the YOLOv11 model with a dedicated dataset to achieve high-precision real-time personnel detection; paired it with the ByteTrack algorithm to complete multi-target tracking, and then used the fine-tuned S3D network to identify children's dangerous actions or abnormal behaviors-grade these behaviors according to their danger levels and trigger corresponding preliminary early warnings. Then,we transmit these early warning results and scene images to the Qwen3-VL model for scene-level risk reasoning and finally generate a complete safety analysis report. We conducted tests in real kindergarten scenarios, and the results show that this framework can quickly and accurately detect personnel, identify various actions, issue safety early warnings in a timely manner, and complete intelligent scene analysis, meeting the actual usage needs of kindergartens.
Xiaojian Rao, Lin Fan, Yong Tian et al.· International Conference on...· 0 citations
Unsupervised 3D skeleton-based action recognition offers advantages in terms of robustness and computational efficiency. However, prevailing paradigms, such as masked skeleton modeling (MSM) and contrastive learning (CL), have inherent limitations: MSM often learns representations with limited discriminative capability, whereas CL may fail to adequately capture fine-grained structural information. Furthermore, integrating the two paradigms through multi-task learning (MTL) can lead to gradient conflicts that limit performance. To address these issues, we propose MEMC (Masked Modeling with Efficient and Minimal Contrastive Learning), a novel framework that adopts an efficient sequential cascade strategy based on layer-grafted pretraining. MEMC first uses MSM to learn low-level representations and then refines high-level representations through CL, thereby avoiding the gradient conflicts associated with MTL. To improve the discriminative capability of the learned representations, particularly for subtle actions, we introduce two CL enhancements. First, Topology Distance-Aware Chain Motion Modeling incorporates skeletal topology priors to capture discriminative motion patterns along skeletal chains. Second, Frequency Band-Aware Contrastive Learning with Frequency Band Pooling (FBP) separates and integrates high-frequency details with low-frequency global context. Extensive experiments on three benchmark datasets demonstrate the effectiveness of MEMC and show its superior performance compared with state-of-the-art methods.
Ancient murals are invaluable cultural heritage, and their automatic classification is crucial for digital preservation, stylistic analysis, and heritage management. Existing approaches require large, high-quality annotated datasets, but such expert annotations are costly and often unavailable. To address this, we formulate a few-shot ancient mural classification problem and propose a Parameter-Efficient Cross-modal Prompt Tuning (PE-CPT) framework. Specifically, PE-CPT adapts a pre-trained vision-language model by learning a small set of prompt vectors without updating its original parameters. It performs cross-modal prompt tuning between image and text encoders, generating image prompts from text prompts to maintain semantic consistency. To further improve adaptation, we incorporate LoRA, a parameter-efficient adaptation method applied to attention and feed-forward layers. Experiments in a 16-shot setting demonstrate that PE-CPT outperforms state-of-the-art baselines, achieving improvements of 6.79%, 6.59%, and 7.07% in accuracy, precision, and F1 score. These results demonstrate its effectiveness in cultural heritage analysis under data scarcity.
Donglai Fu, Chenlong Wang, Zixuan Li et al.· npj Heritage Science· 0 citations