Multi-label, multi-task CNN approach for context-based emotion recognition - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Information Fusion Année : 2021

Multi-label, multi-task CNN approach for context-based emotion recognition

Résumé

This paper proposes a new deep learning architecture for context-based multi-label multi-task emotion recognition. The architecture is built from three main modules: (1) a body features extraction module, which is a pre-trained Xception network, (2) a scene features extraction module, based on a modified VGG16 network, and (3) a fusion-decision module. Moreover, three categorical and three continuous loss functions are compared in order to point out the importance of the synergy between loss functions when it comes to multi-task learning. Then, we propose a new loss function, the multi-label focal loss (MFL), based on the focal loss to deal with imbalanced data. Experimental results on EMOTIC dataset show that MFL with the Huber loss gave better results than any other combination and outperformed the current state of art on the less frequent labels.
Fichier non déposé

Dates et versions

hal-03106102 , version 1 (11-01-2021)

Identifiants

Citer

Ilyes Bendjoudi, Frédéric Vanderhaegen, Denis Hamad, Fadi Dornaika. Multi-label, multi-task CNN approach for context-based emotion recognition. Information Fusion, 2021, 76, pp.422-428. ⟨10.1016/j.inffus.2020.11.007⟩. ⟨hal-03106102⟩
87 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More