Complementing Brightness Constancy with Deep Networks for Optical Flow Prediction - Archive ouverte HAL Access content directly
Conference Papers Year : 2022

Complementing Brightness Constancy with Deep Networks for Optical Flow Prediction

Abstract

State-of-the-art methods for optical flow estimation rely on deep learning, which require complex sequential training schemes to reach optimal performances on real-world data. In this work, we introduce the COMBO deep network that explicitly exploits the brightness constancy (BC) model used in traditional methods. Since BC is an approximate physical model violated in several situations, we propose to train a physically-constrained network complemented with a data-driven network. We introduce a unique and meaningful flow decomposition between the physical prior and the data-driven complement, including an uncertainty quantification of the BC model. We derive a joint training scheme for learning the different components of the decomposition ensuring an optimal cooperation, in a supervised but also in a semi-supervised context. Experiments show that COMBO can improve performances over state-of-the-art supervised networks, e.g. RAFT, reaching state-of-theart results on several benchmarks. We highlight how COMBO can leverage the BC model and adapt to its limitations. Finally, we show that our semi-supervised method can significantly simplify the training procedure.
Fichier principal
Vignette du fichier
4899.pdf (7.25 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03717740 , version 1 (03-09-2022)

Identifiers

Cite

Vincent Le Guen, Clément Rambour, Nicolas Thome. Complementing Brightness Constancy with Deep Networks for Optical Flow Prediction. ECCV 2022, Oct 2022, Tel Aviv, Israel. ⟨10.1007/978-3-031-19803-8_8⟩. ⟨hal-03717740⟩
99 View
15 Download

Altmetric

Share

Gmail Facebook X LinkedIn More