Learning Actionness via Long-range Temporal Order Verification - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2020

Learning Actionness via Long-range Temporal Order Verification

Résumé

Current methods for action recognition typically rely on supervision provided by manual labeling. Such methods, however, do not scale well given the high burden of manual video annotation and a very large number of possible actions. The annotation is particularly difficult for temporal action localization where large parts of the video present no action, or background. To address these challenges, we here propose a self-supervised and generic method to isolate actions from their background. We build on the observation that actions often follow a particular temporal order and, hence, can be predicted by other actions in the same video. As consecutive actions might be separated by minutes, differently to prior work on the arrow of time, we here exploit long-range temporal relations in 10-20 minutes long videos. To this end, we propose a new model that learns actionness via a self-supervised proxy task of order verification. The model assigns high actionness scores to clips which order is easy to predict from other clips in the video. To obtain a powerful and action-agnostic model, we train it on the large-scale unlabeled HowTo100M dataset with highly diverse actions from instructional videos. We validate our method on the task of action localization and demonstrate consistent improvements when combined with other recent weakly-supervised methods.
Fichier principal
Vignette du fichier
paper.pdf (4.25 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03048753 , version 1 (09-12-2020)

Identifiants

  • HAL Id : hal-03048753 , version 1

Citer

Dimitri Zhukov, Jean-Baptiste Alayrac, Ivan Laptev, Josef Sivic. Learning Actionness via Long-range Temporal Order Verification. ECCV 2020 - European Conference on Computer Vision, Aug 2020, Glasgow / Virtual, United Kingdom. ⟨hal-03048753⟩
112 Consultations
260 Téléchargements

Partager

Gmail Facebook X LinkedIn More