A high-performing teacher is built that makes navigation evidence selection explicit and compressible, and a compact student is trained by transferring both where to attend and what to do, then further match action distributions during fine-tuning.
Zhihao Chen, Yi-Yuan Ge, Zi-Yang Wang et al.· IEEE transactions on circuit...· 0 citations
SLU encompasses a diverse set of tasks, including ISLR, CSLR, and SLT. Although these tasks share basic semantic and linguistic foundations, they are typically addressed with task-specific architectures and training pipelines, which hinders knowledge sharing and requires costly pretraining and finetuning for each task....
Shiwei Gan, Yafeng Yin, Xiao Liu et al.· 0 citations
Developmental screening videos show how children perform specific behaviors, but clinical records usually contain outcomes rather than descriptions of what happened. We present SemReward-VL, which learns to describe item-specific behavior from these outcomes. A vision-language model generates a description, and a froze...
De Jiang, Shuo Zhang, Kehong Yuan et al.· 0 citations
Nutrition estimation is a fundamental task in consumer diet tracking, clinical dietetics, chronic disease management, sports and hospital nutrition, and broader food computing systems. The existing approaches have progressed along two largely separate axes, vision models that rely on calibrated RGB-depth captures and i...
Aman Kumar, Avinash Anand, Chaitanya Lakhchaura et al.· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
Diagnosing a patient from video requires more than recognizing the sign: a vision-language model must turn what it sees into hypotheses, questions and tests. DynamicDx evaluates each step in 71 neurological consultations across 11 sign categories, linking authentic patient videos to confirmed diagnoses and fixed charts...
Jiahui Li, Yutong Guo, Nan Yang et al.· 0 citations
It is found that, given task-specific natural-language guidance, VLMs can estimate actuator-response relationships, resolve ambiguous observations through intervention, and actively create informative visual feedback when signals are sparse, suggesting that pretrained multimodal models can serve as important decision-m...
Ryan Lopez, Sachin Vaidya, Seouing-Ho Choi et al.· 0 citations
State-of-the-art pathology foundation models, trained on millions of histology tiles, can fail to preserve tissue similarity when comparisons cross slide or institution boundaries. We show that general-purpose multimodal LLMs, without being trained as pathology foundation models, consistently outperform these specializ...
The proposed PolyTopoBench evaluates both exterior and interior rings, and benchmarks 11 representative methods, including segmentation-based polygonization pipelines, vision foundation model baselines, and specialized vector polygon generators, on two RS-image datasets covering buildings, roads, vegetation, and unvege...
Ze-Ping Liu, Ni Lao, Wei-Wei Sun et al.· 0 citations
Large vision-language models (LVLMs) inherit massive activations from their text-only bases: spikes where a few fixed hidden channels receive values thousands of times above the typical magnitude. The text spike systematically appears in early layers at a fixed initial position, independently of input content. Visual s...
Jonas Ngnaw\'e, Yann Pequignot, Sabyasachi Sahoo et al.· 0 citations
Existing geospatial vision-language models (Geo-VLMs) typically optimize diverse geospatial tasks through a unified multi-task adaptation paradigm without explicitly accounting for the heterogeneous optimization characteristics. Our empirical observations reveal heterogeneous gradient characteristics across tasks, incl...
Dongdong Wang, Deepak Balakrishnan, Ravi Srinivasan et al.· 0 citations
REALIS provides a unified framework for measuring and improving the reliability of AI-image detectors under conditions that better reflect real-world use, and proposes a robustness protocol covering 35 transformations at five severity levels to analyze detector behavior under image processing.
A. Gushchin, Khaled Abud, G. Bychkov et al.· 0 citations
Recent multimodal large language models (MLLMs) advance visual reasoning by strengthening both perception and reasoning, implicitly assuming a process that transitions seamlessly from perception to reasoning. However, we observe a counterintuitive phenomenon that challenges this assumption: holding the model, question,...
Jin-Tao Tong, Yujing Lou, Zhan-Ming Shen et al.· 0 citations
Exploring how generative AI could make machine vision more accessible to businesses. The post GenEye in a Box: Making Machine Vision Something You Can Just Ask For appeared first on GPT-Lab.
Jennifer Neville did not want to go into computer science—but that’s exactly where she landed. Neville discusses the starts and stops that led to her professional sweet spot and her work identifying “surprising failures” making it hard for AI to handle complexity. The post What AI gets wrong and what failure teaches us appeared first on Microsoft Research.
Computer scientist, entrepreneur, and philanthropist will collaborate with the MIT Schwarzman College of Computing to advance AI and scientific discovery.