Digging into Radiance Grid for Real-Time View Synthesis with Detail Preservation - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

Digging into Radiance Grid for Real-Time View Synthesis with Detail Preservation

Résumé

Neural Radiance Fields (NeRF) [30] series are impressive in representing scenes and synthesizing high-quality novel views. However, most previous works fail to preserve texture details and suffer from slow training speed. A recent method SNeRG [10] demonstrates that baking a trained NeRF as a Sparse Neural Radiance Grid enables real-time view synthesis with slight scarification of rendering quality. In this paper, we dig into the Radiance Grid representation and present a set of improvements, which together result in significantly boosted performance in terms of both speed and quality. First, we propose an HieRarchical Sparse Radiance Grid (HrSRG) representation that has higher voxel resolution for informative spaces and fewer voxels for other spaces. HrSRG leverages a hierarchical voxel grid building process inspired by [29], and can describe a scene at high resolution without excessive memory footprint. Furthermore, we show that directly optimizing the voxel grid leads to surprisingly good texture details in rendered images. This direct optimization is memory-friendly and requires multiple orders of magnitude less time than conventional NeRFs as it only involves a tiny MLP. Finally, we find that a critical factor that prevents fine details restoration is the misaligned 2D pixels among images caused by camera pose errors. We propose to use the perceptual loss to add tolerance to misalignments, leading to the improved visual quality of rendered images.
Fichier principal
Vignette du fichier
ReGrid_ECCV2022.pdf (13.13 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04434386 , version 1 (02-02-2024)

Identifiants

  • HAL Id : hal-04434386 , version 1

Citer

Jian Zhang, Jinchi Huang, Bowen Cai, Huan Fu, Mingming Gong, et al.. Digging into Radiance Grid for Real-Time View Synthesis with Detail Preservation. European Conference on Computer Vision, 2022, Oct 2022, Tel Aviv, Israel. ⟨hal-04434386⟩
4 Consultations
5 Téléchargements

Partager

Gmail Facebook X LinkedIn More