Review of Key Technologies and Application Scenarios of Neural Rendering

Abstract

At prohibitive labor costs and low flexibility, physically based rendering is far from effective for the high-fidelity 3D content creation, thus in this paper, we attempt to give a survey of the area of neural rendering. We explain how deep neural networks learn implicit representation of scenes from visual data for efficient and photorealistic novel view synthesis. We review three common types of neural rendering methods: implicit representation (such as neural radiance fields (NeRF) and its subsequent architectures such as Instant-NGP), explicit representation (such as neural texture), and hybrids (such as NeuS and 3D Gaussian splatting). We explain their essence, linkages, advantages and disadvantages. For example, 3D Gaussian splatting achieves state-of-the-art results on the celebrated Blender dataset (33.55 dB PSNR, 0.967 SSIM) with real-time rendering 134 FPS, solving the efficiency limit of vanilla NeRF.Looking ahead, the combination of neural rendering with generative AI, large-scale 3D foundation models and lightweight architectures will lay a foundation for the emerging immersive internet, and digital twin landscape.