A Multi-modal Visual Emotion Recognition Method to Instantiate an Ontology - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2021

A Multi-modal Visual Emotion Recognition Method to Instantiate an Ontology

Résumé

Human emotion recognition from visual expressions is an important research area in computer vision and machine learning owing to its significant scientific and commercial potential. Since visual expressions can be captured from different modalities (e.g., face expressions, body posture, hands pose), multi-modal methods are becoming popular for analyzing human reactions. In contexts in which human emotion detection is performed to associate emotions to certain events or objects to support decision making or for further analysis, it is useful to keep this information in semantic repositories, which offers a wide range of possibilities for implementing smart applications. We propose a multi-modal method for human emotion recognition and an ontology-based approach to store the classification results in EMONTO, an extensible ontology to model emotions. The multi-modal method analyzes facial expressions, body gestures, and features from the body and the environment to determine an emotional state; this processes each modality with a specialized deep learning model and applying a fusion method. Our fusion method, called EmbraceNet+, consists of a branched architecture that integrates the EmbraceNet fusion method with other ones. We experimentally evaluate our multi-modal method on an adaptation of the EMOTIC dataset. Results show that our method outperforms the single-modal methods.
Fichier principal
Vignette du fichier
A Multi-modal Visual Emotion Recognition Method to Instantiate an Ontology.pdf (1.25 Mo) Télécharger le fichier
Origine : Fichiers éditeurs autorisés sur une archive ouverte

Dates et versions

hal-03298743 , version 1 (27-07-2021)

Identifiants

Citer

Juan Heredia, Yudith Cardinale, Irvin Dongo, Jose Díaz-Amado. A Multi-modal Visual Emotion Recognition Method to Instantiate an Ontology. 16th International Conference on Software Technologies, Jul 2021, Online Streaming, France. pp.453-464, ⟨10.5220/0010516104530464⟩. ⟨hal-03298743⟩
184 Consultations
207 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More