JOADAA: joint online action detection and action anticipation - Archive ouverte HAL
Communication Dans Un Congrès Année : 2024

JOADAA: joint online action detection and action anticipation

Mohammed Guermal
Rui Dai
  • Fonction : Auteur
  • PersonId : 1057956
Abid Ali
  • Fonction : Auteur
  • PersonId : 1117945

Résumé

Action anticipation involves forecasting future actions by connecting the past events to future ones. However, this reasoning ignores the real-life hierarchy of events which is considered to be of three main parts: past, present, and future. We argue that considering these three main parts and their dependencies could improve performance. On the other hand, online action detection is the task of predicting actions in a streaming manner. In this case, one has access only to the past and present information. Therefore, in online action detection (OAD) the existing approaches miss semantics or future information which limits the performance of existing approaches. To sum up, for both of these tasks, the complete set of knowledge (past-present-future) is missing, which makes it challenging to infer action dependencies achieving good performances. To address this limitation, we propose fusing both tasks in one uniform architecture. By combining action anticipation and online action detection, our approach can cover the missing dependencies of future information in online action detection. This method, referred as JOADAA, presents a uniform model that jointly performs action anticipation and online action detection. We validate our proposed model on three challenging datasets: THUMOS, which is a sparsely annotated dataset with one action per time step, CHARADES and Multi-THUMOS, two densely annotated datasets, with more complex scenarios. JOADAA achieves SOTA results on these benchmarks for both tasks.
Fichier principal
Vignette du fichier
Guermal_JOADAA_Joint_Online_Action_Detection_and_Action_Anticipation_WACV_2024_paper.pdf (553.42 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04516371 , version 1 (22-03-2024)

Identifiants

  • HAL Id : hal-04516371 , version 1

Citer

Mohammed Guermal, Rui Dai, Francois Bremond, Abid Ali. JOADAA: joint online action detection and action anticipation. WACV 2024 - IEEE/CVF Winter Conference on Applications of Computer Vision, Jan 2024, Waikoloa (Hawaii), United States. ⟨hal-04516371⟩
27 Consultations
27 Téléchargements

Partager

More