FirstPiano: A New Egocentric Hand Action Dataset Oriented Towards Augmented Reality Applications - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

FirstPiano: A New Egocentric Hand Action Dataset Oriented Towards Augmented Reality Applications

Résumé

Research on hand action recognition has achieved very interesting performance in recent years, notably thanks to deep learning methods. With those improvements, we can see new visions towards real appli- cations of new Human-Machine interfaces (HMI) using this recognition. Such new interactions and interfaces need data to develop the best user experience iteratively. However, current datasets for hand action recognition in an egocentric view, even if perfectly useful for these problems of recognition, they generally lack of a limited but coherent context for the proposed actions. Indeed, these datasets tend to provide a wide range of actions, more or less in relation to each other, which does not help to create an interesting context for HMI application purposes. Thereby, we present in this paper a new dataset, FirstPiano, for hand action recognition in an egocentric view, in the context of piano training. FirstPiano provides a total of 672 video sequences directly extracted from the sen- sors of the Microsoft HoloLens Augmented Reality device. Each sequence is provided in depth, infrared and grayscale data, with 4 different points of view for the last one, for a total of 6 streams for each video. We also present the first benchmark of experiments using a Capsule Network over different classification problems and different stream combinations. Our dataset and experiments can therefore be interesting for research communities of action recognition and human-machine interface.
Fichier principal
Vignette du fichier
ICIAP2022-VoilleminEtAl-perso.pdf (613.53 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03982965 , version 1 (11-07-2023)

Identifiants

Citer

Théo Voillemin, Hazem Wannous, Jean-Philippe Vandeborre. FirstPiano: A New Egocentric Hand Action Dataset Oriented Towards Augmented Reality Applications. 21st International Conference on Image Analysis and Processing (ICIAP 2022), May 2022, Lecce, Italy. pp.170-181, ⟨10.1007/978-3-031-06433-3_15⟩. ⟨hal-03982965⟩
26 Consultations
19 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More