Multi-stage RGB-based Transfer Learning Pipeline for Hand Activity Recognition - Archive ouverte HAL Access content directly
Conference Papers Year :

Multi-stage RGB-based Transfer Learning Pipeline for Hand Activity Recognition

Pipeline d'apprentissage par transfert multi-étapes basé sur des séquences d'images RVB pour la reconnaissance de l'activité des mains

Catherine Soladie
Nam-Duong Duong
Jérôme Royan
Renaud Seguier

Abstract

First-person hand activity recognition is a challenging task, especially when not enough data are available. In this paper, we tackle this challenge by proposing a new low-cost multi-stage learning pipeline for firstperson RGB-based hand activity recognition on a limited amount of data. For a given RGB image activity sequence, in the first stage, the regions of interest are extracted using a pre-trained neural network (NN). Then, in the second stage, high-level spatial features are extracted using pre-trained deep NN. In the third stage, the temporal dependencies are learned. Finally, in the last stage, a hand activity sequence classifier is learned, using a post-fusion strategy, which is applied to the previously learned temporal dependencies. The experiments evaluated on two real-world data sets shows that our pipeline achieves the state-of-the-art. Moreover, it shows that the proposed pipeline achieves good results on limited data.

Dates and versions

hal-03582086 , version 1 (21-02-2022)

Identifiers

Cite

Yasser Mohamed Boutaleb, Catherine Soladie, Nam-Duong Duong, Jérôme Royan, Renaud Seguier. Multi-stage RGB-based Transfer Learning Pipeline for Hand Activity Recognition. 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP), Feb 2022, Online Streaming, France. pp.839-848, ⟨10.5220/0010856200003124⟩. ⟨hal-03582086⟩
93 View
0 Download

Altmetric

Share

Gmail Facebook Twitter LinkedIn More