Learning to Estimate Two Dense Depths from LiDAR and Event Data - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Learning to Estimate Two Dense Depths from LiDAR and Event Data

Résumé

Event cameras do not produce images, but rather a continuous flow of events, which encode changes of illumination for each pixel independently and asynchronously. While they output temporally rich information, they lack any depth information which could facilitate their use with other sensors. LiDARs can provide this depth information, but are by nature very sparse, which makes the depth-to-event association more complex. Furthermore, as events represent changes of illumination, they might also represent changes of depth; associating them with a single depth is therefore inadequate. In this work, we propose to address these issues by fusing information from an event camera and a LiDAR using a learning-based approach to estimate accurate dense depth maps. To solve the "potential change of depth" problem, we propose here to estimate two depth maps at each step: one "before" the events happen, and one "after" the events happen. We further propose to use this pair of depths to compute a depth difference for each event, to give them more context. We train and evaluate our network, ALED, on both synthetic and real driving sequences, and show that it is able to predict dense depths with an error reduction of up to 61% compared to the current state of the art. We also demonstrate the quality of our 2-depths-toevent association, and the usefulness of the depth difference information. Finally, we release SLED, a novel synthetic dataset comprising events, LiDAR point clouds, RGB images, and dense depth maps.
Fichier principal
Vignette du fichier
2302.14444.pdf (2.16 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04089239 , version 1 (04-05-2023)

Identifiants

Citer

Vincent Brebion, Julien Moreau, Franck Davoine. Learning to Estimate Two Dense Depths from LiDAR and Event Data. 22nd Scandinavian Conference on Image Analysis (SCIA 2023), Apr 2023, Sirkka, Finland. pp.517-533, ⟨10.1007/978-3-031-31438-4_34⟩. ⟨hal-04089239⟩
49 Consultations
37 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More