Vanishing Point Aided Hash-Frequency Encoding for Neural Radiance Fields (NeRF) from Sparse 360°Input
Résumé
Neural Radiance Fields (NeRF) enable novel view synthesis of 3D
scenes when trained with a set of 2D images. One of the key compo-
nents of NeRF is the input encoding, i.e. mapping the coordinates to
higher dimensions to learn high-frequency details, which has been
proven to increase the quality. Among various input mappings, hash
encoding is gaining increasing attention for its efficiency. However,
its performance on sparse inputs is limited. To address this limitation,
we propose a new input encoding scheme that improves hash-based
NeRF for sparse inputs, i.e. few and distant cameras, specifically for
360◦ view synthesis. In this paper, we combine frequency encoding
and hash encoding and show that this combination can increase
dramatically the quality of hash-based NeRF for sparse inputs. Addi-
tionally, we explore scene geometry by estimating vanishing points
in omnidirectional images (ODI) of indoor and city scenes in order
to align frequency encoding with scene structures. We demonstrate
that our vanishing point-aided scene alignment further improves
deterministic and non-deterministic encodings on image regression
and NeRF tasks where sharper textures and more accurate geometry
of scene structures can be reconstructed.
Domaines
Informatique [cs]Origine | Fichiers produits par l'(les) auteur(s) |
---|