Developing and evaluating automated deep learning and human-in-the-loop vision–language systems for microplastic characterization

Detection-Guided Attention Steering for Vision Language Models

Benchmarking Pre-Trained Feature Extractors: A Comparative Study Across Deep Learning Tasks

This study systematically compares seven pre-trained feature extractors across three architectural families, convolutional neural networks (CNNs), Vision Transformers (ViTs), and self-supervised models to provide practical guidance on model selection for downstream deep learning tasks.

The Evolution of Image Segmentation from Classical Techniques to Deep Learning: A Survey

Image segmentation is a fundamental step in computer vision and a cornerstone of robotic perception, serving as the foundation for interpreting data acquired from vision sensors, enabling robots to analyze complex visual environments, identify and localize objects, and support intelligent decision-making and autonomous control. It plays a critical role in applications such as autonomous navigation, robotic manipulation, medical robotics, agricultural robotics, autonomous vehicles, and human–robot interaction. Image segmentation has evolved from classical methods, which relied on handcrafted rules and mathematical models, to deep learning approaches that learn complex visual patterns directly from data. This evolution reflects advances in algorithms, computational power, and the theoretical foundations of mathematics and data science. Modern deep learning methods rely heavily on large, well-annotated datasets to train sophisticated neural networks. Yet, classical techniques remain valuable in certain scenarios, offering faster, reliable results without extensive computational requirements. Understanding the strengths and limitations of both approaches is key to selecting the right method. This paper surveys image segmentation techniques, comparing them in terms of accuracy, computational cost, and processing speed to guide informed method selection.

SEDCLIP: Adapting vision-language model for multi-label surgical error detection.

Automated detection of surgical errors can drastically increase safety in robotic-assisted surgery (RAS). Prior work has shown surgical error detection only as a binary task, which is insufficient and restrictive when multiple, co-occurring errors inherently arise in real RAS procedures. We thus present the first work to study multi-label surgical error detection (MLSED) and propose SEDCLIP, a novel vision-language adaptation paradigm that transfers pretrained knowledge from a contrastive language-image pretraining model to MLSED. The SEDCLIP architecture comprises: (1) global-local prompting, which aligns global image features and fine-grained patch features with learnable global and local text prompts, countering the single-label bias of contrastive pre-training; (2) a graph-attention temporal adaptor that models inter-frame dependencies, guided by a temporal-distance adjacency; and (3) random-basis low-rank adaptation, which aggregates fixed low-rank random bases modulated by learnable scaling, to bridge the natural-to-surgical domain gap with efficient fine-tuning. Additionally, to catalyse progress in MLSED, we release two first-of-their-kind, in-vivo datasets with frame-level error annotations: MLE-RAH for robotic-assisted hysterectomy (RAH) and MLE-RARP for robotic-assisted radical prostatectomy (RARP). Across both datasets, SEDCLIP yields overall improvements over competing methods on both multi-label and binary surgical error detection tasks. Code and datasets are available at https://github.com/wzjialang/SEDCLIP.

Semisupervised Adaptation of Vision-Language Models for Image Classification

Results on the UC Merced (UCM) and NWPU benchmarks indicate that SE-CLIP significantly outperforms existing semi-supervised approaches and provides a viable solution for adapting VLMs to the remote sensing domain with minimal human intervention.

Argus-Unified: Towards A Compact and Economical Unified Model for Image Understanding and Generation

Unifying visual understanding and generation in one model holds immense promise, but remains challenging and expensive due to heavy compute and data demands and conflicts between the visual features needed for these two capabilities. To address these challenges, we present Argus-Unified, a compact, effective and unified multimodal model built with low demand on computation and data. Instead of aligning modalities from scratch, Argus-Unified effectively leverages pretrained vision-language models (VLMs) that provide strong multimodal priors. Specifically, we introduce hybrid visual tokens that preserve continuous tokens for understanding while learning discrete tokens for generation from a frozen unified vision encoder. Our training pipeline includes two stages: the first stage learns a quantizer and image decoder on top of the frozen vision encoder, the second stage trains the LLM initialized from a pretrained VLM for the unified multimodal modeling. Using by far the least amount of data (15.6M) and the lowest cost (~$2,000), we demonstrate that unified multimodal models can be trained economically while achieving strong performance in both understanding and generation. Notably, our model attains state-of-the-art multimodal understanding on GQA, POPE, and VQAv2, and competitive generation quality compared to models with dedicated vision encoders (e.g., Janus, Janus-Pro), all at ~10x lower cost and with ~5x less data. We envision Argus-Unified as a useful baseline that lowers the development barrier for unified models.