Enhancing Spatial Understanding in Vision-Language Models via Curriculum Learning

Abstract

The development of Embodied AI urgently necessitates high-fidelity environment modeling enriched with spatial context. However, existing 3D semantic scene understanding methods predominantly focus on isolated instance-level labels or high-dimensional semantic vector embeddings, lacking effective modeling of explicit spatial-semantic relationships between objects within a scene. To address this issue, we use a method that guides Vision-Language Models(VLMs) to learn scene spatial layout relationships via Supervised Fine-Tuning (SFT). First, based on the InteriorGS dataset, we construct a Visual Question Answering (VQA) dataset spanning 100 scenes, comprising RGB images and grounding images with 2D bounding box prompts. Within this dataset, we systematically annotate the spatial-semantic relationship graphs among visible instances. Second, we utilize the parameter-efficient fine-tuning strategy of Low-Rank Adaptation (LoRA) to enhance the spatial relationship reasoning capabilities of the baseline model, Qwen2.5-VL-7B-Instruct. Furthermore, we design an easy-to-hard, three-stage curriculum learning scheme: progressing from single-image single-instance relationship reasoning, advancing to single-image multi-instance relationship understanding, and ultimately achieving global spatial layout perception across continuous frames. Comprehensive evaluations demonstrate that our method enables the model to effectively comprehend and output spatial-semantic relationship triplets in a predefined format, significantly outperforming the baseline model in inter-instance spatial-semantic reasoning. Our research validates the feasibility of endowing existing VLMs with preliminary spatial intelligence via SFT, laying the foundation for constructing next-generation semantic scene representations enriched with spatial relational information.