Skip to content

Autoregressive Image to 3D Generation via Next Symmetric View Prediction

Aug 2026 · IEEE Transactions on Pattern Analysis and Machine Intelligence · Vol 48, pp. 15703-15715 · 0 citations · 79 references
Medicine

Abstract

Autoregressive models via next token prediction have emerged as a promising way towards unified models across multiple modalities for general artificial intelligence. Nevertheless, exploiting this idea to achieve single image to 3D generation remains a challenging problem. In this paper, we introduce a new Autoregressive 3D diffusion model (dubbed as AR3D), that novelly frames autoregressive modeling on multi-view images of 3D content as “next symmetric view prediction.” This methodology first triggers 3D autoregressive modeling by remoulding video diffusion model with unidirectional next-view prediction, where the next-view images are iteratively generated conditioned on the past views and input image. Furthermore, a unique autoregressive scheme via next symmetric view prediction is introduced to simultaneously predict each pair of views under the view-order symmetry at each autoregressive step. Such design captures the view-order symmetry of 360<inline-formula><tex-math notation="LaTeX">$^{\circ }$</tex-math><alternatives><mml:math><mml:msup><mml:mrow/><mml:mo>∘</mml:mo></mml:msup></mml:math><inline-graphic xlink:href="pan-ieq1-3726806.gif"/></alternatives></inline-formula> orbital multi-view images and thus encourages strong geometric and appearance consistency among adjacent views in a near-to-far order. Extensive experiments demonstrate that our proposed AR3D achieves strong performance on both novel view synthesis and single view reconstruction tasks for image-to-3D generation.

View source

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.