Arm pose estimation enables applications in fitness, extended reality input, rehabilitation, and life logging. Prior smartwatch-based approaches rely on calibration poses and preprocessing pipelines that transform raw IMU measurements into standardized training formats. These steps hinder deployment in everyday setting...
Bishnu Dev, Vasco Xu, Xi-Aan Loh et al.· 0 citations
DriveMotion is introduced, a multi-source benchmark for continuous driver motion forecasting, integrating naturalistic driving data, curated public in-cabin videos, and the AIDE dataset into a unified representation with per-joint validity masks and synchronized driving context.
Yu-Hang Wang, Chu-Heng Wei, Jing-Xin Yang et al.· 1 citation
Automated assessment of depression from facial dynamics holds promise for scalable mental health monitoring, yet cross-corpus generalization of learned biomarkers remains an open challenge. We present a systematic bidirectional transfer study pairing the EmpkinS-EKSpression randomized controlled trial (RCT; N = 256, SC...
Misha Sadeghi, R. Richer, L. H. Rupp et al.· 0 citations
MM-SVGEdit is proposed, a multimodal-driven SVG editing approach that integrates traditional SVG editing and LLM-based methods and introduces a two-stage strategy in which visual grounding is followed by modification.
This work introduces NEST (Narrative Event Structures in Time for Long Video Understanding), a dataset of 1,005 full-length movies annotated with about 103 multimodal narrative events grounded in visual content, dialogue, and audio, and introduces baselines for event trigger detection, event localization, event argumen...
Ali Asgarov, Kaushik Narasimhan, Najibul Haque Sarker et al.· arXiv.org· 1 citation
Text-to-image diffusion models, such as Stable Diffusion, can produce high-quality and diverse images but often fail to achieve compositional alignment, particularly when prompts describe complex object relationships, attributes, or spatial arrangements. Recent inference-time approaches address this by optimizing or ex...
Seyed Amir Kasaei, Ali Aghayari, Arash Marioriyad et al.· 0 citations
Currently, a prevalent approach for enhancing Vision-Language Models (VLMs) performance is to encode both the high-resolution version and the thumbnail of an image simultaneously. While effective, this method generates a large number of image tokens. When combined with the widely used Rotary Position Embedding (RoPE),...
Chart question-answering (QA) benchmarks aim to pose questions that require visual reasoning to correctly answer, but vision-language models (VLMs) can often reach solutions through shortcuts or prior familiarity with a chart or question. To strictly evaluate visual reasoning, we propose counterfactual charts where the...
Yifan Jiang, Dae Yon Hwang, Jesse C. Cresswell et al.· 0 citations
A controlled pure-autoregressive testbed is built and track task-specific validation losses during multimodal continual pretraining across text, image, text-to-image (T2I), and image-to-text (I2T) prediction, showing that better reconstruction does not necessarily yield lower task-specific losses or stronger downstream...
Siting Li, Zheng-Yang Wang, S. Du et al.· 0 citations
This work presents an RS-specific formulation of the region selection paradigm, previously explored in natural-image MLLMs, and extends it to temporal change localization over multi-image sequences, and constructs a multi-task training and evaluation suite spanning localization, referring expression, visual grounding,...
J. Chung, Sungjune Park, Yeongyun Kim et al.· 0 citations
Token-Adaptive Decoding is proposed, a training-free strategy for hallucination mitigation that grounds the initial yes/no decision by contrasting logits under real audio with a matched silent reference, and introduces a token-adaptive, confidence-guided gate that is decision-critical at the first decoding step.
SignDino, a self-supervised sign-video encoder that moves the DINOv3 student--teacher recipe from the spatial domain of image crops to the temporal domain of tracked sign streams, provides a strong public self-supervised representation and shows competitive or state-of-the-art performance under matched downstream evalu...
Jun-Yi Hu, Zhe-Wen He, Hao Huang et al.· 0 citations
Exploring how generative AI could make machine vision more accessible to businesses. The post GenEye in a Box: Making Machine Vision Something You Can Just Ask For appeared first on GPT-Lab.
Jennifer Neville did not want to go into computer science—but that’s exactly where she landed. Neville discusses the starts and stops that led to her professional sweet spot and her work identifying “surprising failures” making it hard for AI to handle complexity. The post What AI gets wrong and what failure teaches us appeared first on Microsoft Research.
Computer scientist, entrepreneur, and philanthropist will collaborate with the MIT Schwarzman College of Computing to advance AI and scientific discovery.