Secrets of Event-based Optical Flow, Depth and Ego-motion Estimation by Contrast Maximization - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue IEEE Transactions on Pattern Analysis and Machine Intelligence Année : 2024

Secrets of Event-based Optical Flow, Depth and Ego-motion Estimation by Contrast Maximization

Résumé

Event cameras respond to scene dynamics and provide signals naturally suitable for motion estimation with advantages, such as high dynamic range. The emerging field of event-based vision motivates a revisit of fundamental computer vision tasks related to motion, such as optical flow and depth estimation. However, state-of-the-art event-based optical flow methods tend to originate in frame-based deep-learning methods, which require several adaptations (data conversion, loss function, etc.) as they have very different properties. We develop a principled method to extend the Contrast Maximization framework to estimate dense optical flow, depth, and ego-motion from events alone. The proposed method sensibly models the space-time properties of event data and tackles the event alignment problem. It designs the objective function to prevent overfitting, deals better with occlusions, and improves convergence using a multi-scale approach. With these key elements, our method ranks first among unsupervised methods on the MVSEC benchmark and is competitive on the DSEC benchmark. Moreover, it allows us to simultaneously estimate dense depth and ego-motion, exposes the limitations of current flow benchmarks, and produces remarkable results when it is transferred to unsupervised learning settings. Along with various downstream applications shown, we hope the proposed method becomes a cornerstone on event-based motion-related tasks. Code is available at https://github.com/tub-rip/event_based_optical_flow
Fichier principal
Vignette du fichier
2024_TPAMI_Secrets_MCM.pdf (25.01 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04655247 , version 1 (21-07-2024)

Licence

Identifiants

Citer

Shintaro Shiba, Yannick Klose, Yoshimitsu Aoki, Guillermo Gallego. Secrets of Event-based Optical Flow, Depth and Ego-motion Estimation by Contrast Maximization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024, pp.1-18. ⟨10.1109/TPAMI.2024.3396116⟩. ⟨hal-04655247⟩
45 Consultations
11 Téléchargements

Altmetric

Partager

Gmail Mastodon Facebook X LinkedIn More