MonoScene: Monocular 3D Semantic Scene Completion - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

MonoScene: Monocular 3D Semantic Scene Completion

Résumé

MonoScene proposes a 3D Semantic Scene Completion (SSC) framework, where the dense geometry and semantics of a scene are inferred from a single monocular RGB image. Different from the SSC literature, relying on 2.5 or 3D input, we solve the complex problem of 2D to 3D scene reconstruction while jointly inferring its semantics. Our framework relies on successive 2D and 3D UNets bridged by a novel 2D-3D features projection inspiring from optics and introduces a 3D context relation prior to enforce spatio-semantic consistency. Along with architectural contributions, we introduce novel global scene and local frustums losses. Experiments show we outperform the literature on all metrics and datasets while hallucinating plausible scenery even beyond the camera field of view. Our code and trained models are available at https://github.com/cv-rits/MonoScene

Dates et versions

hal-03498508 , version 1 (21-12-2021)

Identifiants

Citer

Anh-Quan Cao, Raoul de Charette. MonoScene: Monocular 3D Semantic Scene Completion. Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2022, New orleans, USA, United States. ⟨hal-03498508⟩

Collections

INRIA INRIA2
94 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More