TOWARD SPATIAL INTELLIGENCE AND PHYSICALLY REALISTIC WORLD MODELS

Abstract

Spatial intelligence requires models to infer and maintain coherent three-dimensional structure, preserve objects and relations across viewpoints, compose interactable environments, and respect geometric and physical constraints. Progress toward such world models is limited by two forms of data scarcity: a shortage of diverse, high-quality real-world spatial observations and the narrow distributions represented by curated interactive 3D scene datasets. This dissertation studies how structured data and reusable priors can address both bottlenecks.The first contribution, DL3DV-10K, establishes a high-quality corpus of 10,510 diverse real-world scenes and 51.2 million frames captured as multiview videos, together with a reproducible data-production protocol. Its benchmark reveals novel view synthesis failure modes missed by smaller collections; its scaling experiments show that broader real-world data improves generalizable 3D representation learning; and its adoption across 3D vision, generative video, and hybrid 3D--video systems demonstrates its value as shared infrastructure.The second and third contributions study complementary routes to interactive 3D scene generation beyond curated distributions. Scenethesis grounds language planning and visual guidance with explicit 3D assets and geometric and physical constraints, producing editable indoor and outdoor scenes with more coherent relations, support, and collision behavior. I-Scene instead repurposes a pretrained 3D instance generator as a feed-forward scene-level spatial learner. Learning from non-semantic random compositions reduces its dependence on annotated semantic layouts and supports generalization to unseen object arrangements.Together, these works show that progress toward spatially intelligent and physically realistic world models depends not only on model design but also on how spatial experience is collected, structured, grounded, and reused. High-quality multiview data supplies transferable observations, while inference-time and learning-based approaches expand interactive 3D environment generation beyond bounded datasets. These findings provide foundations for persistent, interactive, and physically grounded world models.