Skip to content

CapGeo-Bench: Decoupling Visual Perception from Reasoning and Evaluating Geometric Understanding

Oct 2025 · 5 citations · 45 references
Computer Science

TL;DR

CapGeo-Bench is proposed, a benchmark of 4,641 high-quality figure-caption pairs equipped with a fine-grained keypoint-based evaluation metric that provides high-quality captions consistently and substantially boosts performance of MLLMs, empirically validating the visual perception bottleneck in geometric reasoning.

Abstract

While Multimodal Large Language Models (MLLMs) have achieved remarkable success in difficult purely textual mathematical reasoning tasks, even advanced closed-source models such as GPT-o3 still struggle with geometric problems. This discrepancy motivates us to investigate the root cause: is the bottleneck of multimodal geometric reasoning rooted in reasoning itself, or in the perception of geometric information from diagrams? To answer this question, we conduct an exploratory experiment and find that providing high-quality captions consistently and substantially boosts performance of MLLMs, empirically validating the visual perception bottleneck in geometric reasoning. However, MLLMs'capabilities in visual geometric perception remain insufficiently evaluated. To fill the gap, we propose CapGeo-Bench, a benchmark of 4,641 high-quality figure-caption pairs equipped with a fine-grained keypoint-based evaluation metric. Specifically, CapGeo-Bench encompasses three geometric classes and is divided into four difficulty levels according to the complexity of geometric primitives. Captions are scored at a fine-grained level by extracting keypoints along three dimensions-geometric elements, spatial relationships, and numerical attributes. CapGeo-Bench provides a reliable instrument and guideline for evaluating and advancing MLLMs'geometric perception capabilities. Our code and data are publicly available at: https://github.com/YuYingLi0/CapGeo.

View source

Similar papers

Preprint Oct 2026

Diagnosing the Reference Gap: What Visual Scaffolds Can and Cannot Fix in Multimodal Reasoning

Multimodal large language models can recognize visual content yet struggle to reason reliably over it across multiple steps. In counting, a model may see all objects but lose track of which ones it has counted. We argue that such failures, commonly attributed to perception, reveal a reference gap: the failure to bind l...

Gui-Zhen Chen, Wei-Wen Xu, Wo-Tao Yin et al. · 0 citations
Book Open access Nov 2026

Beyond Answer Accuracy: Diagnosing Perception Failures in Multimodal Mathematical Reasoning

Multimodal Large Language Models (MLLMs) have shown strong performance on mathematical reasoning tasks that combine textual and visual information. However, final-answer accuracy offers limited insight into whether errors arise from incorrect perception, modality-dependent information extraction, or downstream reasonin...

Aniket Mitra, Arpit Chhabra, Aditya Desai et al. · 0 citations
#artificial intelligence Preprint Nov 2026

Seeing Isn't Enough: Unlocking Visual Reasoning by Bridging the Reasoning-Access Gap with TIGER

Multimodal Large Language Models (MLLMs) inherit strong textual reasoning capabilities from their language backbones and text pretraining, but often fail to apply the same level of competency to visual inputs. We show that this degradation cannot be attributed to only perception failures, by evaluating models on semant...

G. Lau, Minh Huynh Nguyen, B. Low · 0 citations
#artificial intelligence Preprint Oct 2026

Where Do Vision Language Models Fail at 3D Reasoning?

Vision-language models can get individual judgments right while struggling to recover and consistently use the exact properties of a 3D object. To characterize this gap, we introduce BedBench, a controlled benchmark of thirteen procedurally generated tasks spanning individual judgments and composite answers. Across twe...

Khai Le-Duc, Nguyen Kim Hai Bui, An Ngo et al. · 1 citation
#artificial intelligence Preprint Oct 2026

TriGeoBench: A Multimodal Mathematical Geometry Benchmark with Figures and TikZ

Multimodal Large Language Models show strong vision-language capabilities, yet their geometry-centric reasoning remains insufficiently understood. Existing mathematical benchmarks typically rely on rasterized figures and cover limited geometric knowledge areas and response formats, making it difficult to assess whether...

Jing Wang, Yi-Tong Yang, Hao Dou et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.