Continual Unsupervised Learning for Optical Flow Estimation with Deep Networks - Archive ouverte HAL Access content directly
Conference Papers Year :

Continual Unsupervised Learning for Optical Flow Estimation with Deep Networks


In the last few years there has been a growing interest in approaches that allow neural networks to learn how to predict optical flow, both in a supervised and, more recently, unsupervised manner. While this clearly opens up the possibility of learning to estimate optical flow in a truly lifelong setting, by processing a potentially endless video stream, existing techniques assume to have access to large datasets and they perform stochastic mini-batch-based gradient optimization, paired with further ad-hoc components. We present an extensive study on how neural networks can learn to estimate optical flow in a continual manner while observing a long video stream and reacting online to the streamed information without any further data buffering. To this end, we rely on photo-realistic video streams that we specifically created using 3D virtual environments, as well as on a real-world movie. Our analysis considers important model selection issues that might be easily overlooked at a first glance, comparing different neural architectures and also state-of-the-art models pretrained in an offline manner. Our results not only show the feasibility of continual unsupervised learning in optical flow estimation, but also indicate that the learned models, in several situations, are comparable to state-of-the-art offline-pretrained networks. Moreover, we show how common issues in continual learning, such as catastrophic forgetting, do not affect the proposed models in a disruptive manner, given the task at hand.
Fichier principal
Vignette du fichier
_CoLLAs_2022__Continual_Unsupervised_Online_Learning_of_Optical_Flow.pdf (1.49 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03881291 , version 1 (01-12-2022)


  • HAL Id : hal-03881291 , version 1


Simone Marullo, Matteo Tiezzi, Alessandro Betti, Lapo Faggi, Enrico Meloni, et al.. Continual Unsupervised Learning for Optical Flow Estimation with Deep Networks. CoLLAs 2022 - Conference on Lifelong Learning Agents, Aug 2022, Montreal, Canada. ⟨hal-03881291⟩
12 View
9 Download


Gmail Facebook Twitter LinkedIn More