Skip to content
#computer vision Preprint

Luce: Relightable Gaussians for 3D Asset Generation

Aug 2026 · 0 citations · 56 references
Computer Science

Abstract

High-fidelity image-to-3D generation requires a 3D representation that captures both geometry and appearance. To support relighting and integration into standard rendering pipelines, the representation should include physically based rendering (PBR) modalities such as albedo, metallic-roughness, and surface normals. We propose Luce, a 3D representation that unifies geometry and PBR materials within a voxelized multimodal Gaussian cloud, using dedicated Gaussian primitives for each modality. A variational autoencoder compresses this representation into a unified material-aware latent space. A rectified-flow transformer generates this latent from a single image, conditioned on multi-layer features from a pretrained image encoder that preserve both semantic context and fine spatial detail. The latent then decodes into relightable PBR Gaussians and an optional textured mesh with a tangent-space normal map. On Toys4K, Luce achieves state-of-the-art single-image-to-3D generation, improving FID by 28% over the strongest baseline. We further introduce a benchmark of AI-generated images, on which Luce improves the CLIP image-alignment score over the best baseline (0.8519 vs. 0.8299). Luce generates relightable, geometrically accurate, and materially faithful assets that preserve fine details such as text, logos, and inscriptions.

View source

Similar papers

Preprint Sep 2026

LightBridge: Feed-Forward Generative Relighting for 3D Gaussian Splatting

LightBridge is presented, a feed-forward generative framework for controllable relighting of complete 3DGS assets in a single pass and efficient single-pass prediction of complete relit 3DGS assets without scene-specific optimization.

Heng Cao, Pan-Hao Cheng, Huang-Sheng Du et al. · 0 citations
Sep 2026

OmniFabric: Coherent UV Space Texture Synthesis for 3D Garment Reconstruction

Automated generation of production-ready 3D garment assets from a single image is a central challenge in digital content creation. While recent generative models have significantly advanced 3D geometry reconstruction, synthesizing high-quality textures remains a bottleneck. Existing methods often bake environmental ill...

Di Huang, Yuan-Hao Wang, Cheng Zhang et al. · 0 citations
Preprint Aug 2026

DiGS-Avatar: Single-Image Animatable 3D Human Reconstruction via UV-Space Diffusion

DiGS-Avatar is proposed, which reformulates this task as an efficient, diffusion-based UV-latent completion task, ensuring 3D consistency by design, and introduces a teacher-student framework where a multi-view teacher provides geometrically aligned pseudo-ground-truth latents to supervise a single-view diffusion stude...

Jiakun Li, Li Fang, Hao Zhu et al. · 0 citations
Preprint Aug 2026

LumiTokens: 3D Relighting via Token-Space Lighting Transformation

LumiTokens is a framework that formulates 3D relighting as a direct transformation on latent scene tokens, without explicit 3D representations, rendering equations, or physics-based decomposition, and achieves comparable or superior relighting quality to other methods and supports progressive, composable lighting edits...

Yiwen Chen, Matheus Gadelha, Huai-Zu Jiang · 0 citations
Preprint Aug 2026

SpatialCrafter: Single Image World Modeling with Generative 3D Proxies

SpatialCrafter is presented, a novel two-stage framework that addresses explorable image-to-scene generation issues by introducing a global 3D proxy for high-fidelity image-to-scene generation and appearance refinement and introduces Parallel Geometry Injection and Proxy-Aware Corruption training strategies.

Chuan Fang, Lingteng Qiu, Yixun Liang et al. · 3 citations
Conference Aug 2026

ENEA-GS: Enhancing Single-Image 3D Gaussian Splatting with Normal-guided Texture Smoothness and Entropy-based Alpha Regularization

Recent advances in 3D content generation have demonstrated the effectiveness of optimization-based frameworks such as DreamGaussian, which combine 3D Gaussian Splatting (3DGS) with diffusion-guided Score Distillation Sampling (SDS) to efficiently synthesize 3D assets from a single image. Compared with earlier NeRF-base...

Nam-Quan Nguyen, Lam-Huy Nguyen, Minh-Triet Tran · 0 citations

Related blog posts

MIT News · Artificial Intelligence Oct 28, 2026

Making machines that can listen

Professor Paris Smaragdis develops computers with superhuman hearing for applications like speech recognition, music understanding, and environmental monitoring.

Microsoft Research Blog Oct 6, 2026

What AI gets wrong and what failure teaches us

Jennifer Neville did not want to go into computer science—but that’s exactly where she landed. Neville discusses the starts and stops that led to her professional sweet spot and her work identifying “surprising failures” making it hard for AI to handle complexity.  The post What AI gets wrong and what failure teaches us appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.