Automatic scoring of synchronization from fingers motion capture and music beats - Archive ouverte HAL
Communication Dans Un Congrès Année : 2022

Automatic scoring of synchronization from fingers motion capture and music beats

Hamza Bayd
  • Fonction : Auteur
  • PersonId : 1136238
  • IdHAL : hamza-bayd
Patrice Guyot
Benoit Bardy
Pierre Slangen

Résumé

Thanks to various technological progresses such as musical rhythm estimation and motion capture systems, the evaluation of synchronization performances between motion and music beats is today possible. In this paper, we propose an innovative playful method to assess synchronization between hand motion and music. In this application, the hand gestures are tracked from live webcam motion capture based on MediaPipe open-source framework. Musical beats are estimated from the Librosa library. The synchronization between finger motion and beats is then computed from the dynamic time warping algorithm. A preliminary study was conducted with two different songs mixed with tempo beats at different intensities. The results are encouraging and show different levels of performance according to the tempo of the songs.
Fichier principal
Vignette du fichier
iSHAPE.pdf (615.75 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03682819 , version 1 (31-05-2022)

Identifiants

Citer

Hamza Bayd, Patrice Guyot, Benoit Bardy, Pierre Slangen. Automatic scoring of synchronization from fingers motion capture and music beats. ISHAPE 2022 - 1st Int. Workshop on Intelligent Systems in Human and Artificial Perception, May 2022, Lecce, Italy. pp.235-245, ⟨10.1007/978-3-031-13321-3_21⟩. ⟨hal-03682819⟩
126 Consultations
377 Téléchargements

Altmetric

Partager

More