SCOTCH and SODA: A Transformer Video Shadow Detection Framework - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

SCOTCH and SODA: A Transformer Video Shadow Detection Framework

Résumé

Shadows in videos are difficult to detect because of the large shadow deformation between frames. In this work, we argue that accounting for the shadow deformation is essential when designing a video shadow detection method. To this end, we introduce the shadow deformation attention trajectory (SODA), a new type of video self-attention module, specially designed to handle the large shadow deformations in videos. Moreover, we present a shadow contrastive learning mechanism (SCOTCH) which aims at guiding the network to learn a high-level representation of shadows, unified across different videos. We demonstrate empirically the effectiveness of our two contributions in an ablation study. Furthermore, we show that SCOTCH and SODA significantly outperforms existing techniques for video shadow detection. Code will be available upon the acceptance of this work.

Dates et versions

hal-03944252 , version 1 (17-01-2023)

Identifiants

Citer

Lihao Liu, Jean Prost, Lei Zhu, Nicolas Papadakis, Pietro Liò, et al.. SCOTCH and SODA: A Transformer Video Shadow Detection Framework. IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR'23), Jun 2023, Vancouver, Canada. ⟨hal-03944252⟩

Collections

CNRS
33 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More