Vanishing Point Aided Hash-Frequency Encoding for Neural Radiance Fields (NeRF) from Sparse 360°Input - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Vanishing Point Aided Hash-Frequency Encoding for Neural Radiance Fields (NeRF) from Sparse 360°Input

Résumé

Neural Radiance Fields (NeRF) enable novel view synthesis of 3D scenes when trained with a set of 2D images. One of the key compo- nents of NeRF is the input encoding, i.e. mapping the coordinates to higher dimensions to learn high-frequency details, which has been proven to increase the quality. Among various input mappings, hash encoding is gaining increasing attention for its efficiency. However, its performance on sparse inputs is limited. To address this limitation, we propose a new input encoding scheme that improves hash-based NeRF for sparse inputs, i.e. few and distant cameras, specifically for 360◦ view synthesis. In this paper, we combine frequency encoding and hash encoding and show that this combination can increase dramatically the quality of hash-based NeRF for sparse inputs. Addi- tionally, we explore scene geometry by estimating vanishing points in omnidirectional images (ODI) of indoor and city scenes in order to align frequency encoding with scene structures. We demonstrate that our vanishing point-aided scene alignment further improves deterministic and non-deterministic encodings on image regression and NeRF tasks where sharper textures and more accurate geometry of scene structures can be reconstructed.
Fichier principal
Vignette du fichier
Vanishing-points-hash-encoding.pdf (44.12 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04197185 , version 1 (05-09-2023)

Licence

Paternité

Identifiants

  • HAL Id : hal-04197185 , version 1

Citer

Kai Gu, Thomas Maugey, Sebastian Knorr, Christine Guillemot. Vanishing Point Aided Hash-Frequency Encoding for Neural Radiance Fields (NeRF) from Sparse 360°Input. ISMAR 2023 - 22nd IEEE International Symposium on Mixed and Augmented Reality, Oct 2023, Sydney, Australia. pp.1-10. ⟨hal-04197185⟩
132 Consultations
2 Téléchargements

Partager

Gmail Facebook X LinkedIn More