Hateful meme detection remains a formidable challenge for vision-language models, as existing benchmarks are structurally observational - confounding rhetorical hate mechanisms with target community features and preventing causal evaluation of model vulnerabilities. To address this, we introduce FBHM, a systematically...
UReason is introduced, a benchmark for evaluating reasoning-to-generation alignment in this paradigm, consisting of 2,000 human-curated and human-verified instances spanning five reasoning-intensive tasks: Code, Arithmetic, Spatial, Attribute, and Text, and it is found that decontextualized generation consistently outp...
Cheng Yang, Chufan Shi, Bo Shui et al.· 5 citations
This work introduces EMemBench, a programmatic benchmark generator for evaluating long-term episodic memory of agents through interactive games, and evaluates memory agents with strong LMs/VLMs as backbones, using in-context prompting as baselines.
Xinze Li, Zi-Yue Zhu, Siyuan Liu et al.· arXiv.org· 7 citations· ⚡1
The introduction of MM-BrowseComp, a novel benchmark comprising 400 challenging, hand-crafted questions designed to evaluate multimodal retrieval and reasoning capabilities, is introduced, establishing MM-BrowseComp as a rigorous new standard for the field.
Shilong Li, Xingyuan Bu, Wenjie Wang et al.· arXiv.org· 37 citations· ⚡7
Reach audiences
Advertise in front of researchers, engineers, and readers.
This work demonstrates that a VLM-based pipeline can successfully unlock historical auction catalogs for large-scale automated analysis, and benchmark the methods across different deployment modes ranging from commercial providers to locally hosted, quantized models.
Mathias Zinnen, Alisha Mund, Sabine Lang et al.· 0 citations
Experimental results show that the proposed model can not only recognize characters in document images but also locate word boundaries, removing the need for an extra word segmentation step in a conventional sequential pipeline.
Marry Kong, Rina Buoy, Sovisal Chenda et al.· 0 citations
SpanCalib-VLM is presented, a hybrid dual-system for the SHROOM-Visions Shared Task that combines a multimodal sequence tagger, consisting of XLM-RoBERTa-Large fused with a SigLIP vision encoder via cross-attention, with the fine-tuned generative VLM (Qwen3.5-4B-SHROOM-SFT).
Evaluating four state-of-the-art models finds that placing a role in a semantically unrelated context does not suppress role-linked attributes; instead, cross-role attribute concentration increases (pooled BI $+0.047$).
Shaghayegh Kolli, S. Emami, Moreno D'Incà et al.· 0 citations
This approach converts detailed visual features into descriptive terms, addressing a key challenge in art history, and connects the use of images as data with the semantic concerns of humanists, establishing vision-based computational art history as an area for future growth.
This work introduces GeoAgent, an agentic environment-based benchmark that requires agents to navigate Street View environments to refine their geolocalization through sequential reasoning, and establishes the challenges of embodied navigation and geospatial reasoning.
Arka Mukherjee, Soham Roy, Kartikeya Trivedi et al.· 0 citations
The results support frozen verification as a training signal for evidence selection, while showing that strict boundary precision remains comparatively weaker.
Ming-Wen Zhang, Ji-Sheng Dang, Min-Qiang Yang et al.· 0 citations
SocialReasonBench is introduced, a video multiple-choice QA benchmark for evaluating socially grounded reasoning in scenarios derived from interactive narratives and develops a multi-agent curation pipeline that localizes socially meaningful clips, grounds answer labels in game-state signals, and generates theory-guide...
Zhe-Yu Huang, Zijing Shi, Hao-Zhe Luo et al.· 0 citations
Exploring how generative AI could make machine vision more accessible to businesses. The post GenEye in a Box: Making Machine Vision Something You Can Just Ask For appeared first on GPT-Lab.
Jennifer Neville did not want to go into computer science—but that’s exactly where she landed. Neville discusses the starts and stops that led to her professional sweet spot and her work identifying “surprising failures” making it hard for AI to handle complexity. The post What AI gets wrong and what failure teaches us appeared first on Microsoft Research.
Computer scientist, entrepreneur, and philanthropist will collaborate with the MIT Schwarzman College of Computing to advance AI and scientific discovery.
