Leveraging Self-Attention Mechanism for Deep Learning in Hand-Gesture Recognition System - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue E3S Web of Conferences Année : 2024

Leveraging Self-Attention Mechanism for Deep Learning in Hand-Gesture Recognition System

Résumé

This research addresses the complex challenge of recognizing hand gestures irrespective of the user's body posture, a crucial issue in medical treatment for people with speech impairments and human-machine interfaces where precise gesture interpretation is vital. The aim is to engineer an advanced hand gesture recognition system, effective across various body positions and camera viewpoints. A novel flexible camera arrangement was employed, integrating a CNN-Transformer hybrid model, leveraging the strengths of Convolutional Neural Networks and the self-attention mechanism of Transformers. Developed using Python and the PyTorch deep learning framework, the system focuses on sophisticated image processing techniques. A thorough literature review on gesture recognition systems and multi-view analysis was conducted to inform the development. The system demonstrated exceptional accuracy in recognizing hand gestures in diverse body postures and from multiple camera perspectives, significantly outperforming existing methods. It marked a significant advancement in decoding complex gestures, a key aspect for medical applications and intricate human-machine interactions. This technology is primarily beneficial for people with speech impairments, rehabilitation, and in human-machine interfaces, poised to revolutionize patient care and enhance interaction with advanced machinery and computer systems.
Fichier principal
Vignette du fichier
Leveraging Self-Attention Mechanism for Deep Learning in Hand-Gesture Recognition System.pdf (2.91 Mo) Télécharger le fichier
Origine : Fichiers éditeurs autorisés sur une archive ouverte
Licence : Domaine public

Dates et versions

hal-04503868 , version 1 (14-03-2024)

Identifiants

Citer

Muhamad Amirul Haq, Le Nam Quoc Huy, Muhammad Ridlwan, Ishmatun Naila. Leveraging Self-Attention Mechanism for Deep Learning in Hand-Gesture Recognition System. E3S Web of Conferences, In press, 500, ⟨10.1051/e3sconf/202450001009⟩. ⟨hal-04503868⟩
2 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More