Multi-label, multi-task CNN approach for context-based emotion recognition - Archive ouverte HAL Access content directly
Journal Articles Information Fusion Year : 2021

Multi-label, multi-task CNN approach for context-based emotion recognition

Abstract

This paper proposes a new deep learning architecture for context-based multi-label multi-task emotion recognition. The architecture is built from three main modules: (1) a body features extraction module, which is a pre-trained Xception network, (2) a scene features extraction module, based on a modified VGG16 network, and (3) a fusion-decision module. Moreover, three categorical and three continuous loss functions are compared in order to point out the importance of the synergy between loss functions when it comes to multi-task learning. Then, we propose a new loss function, the multi-label focal loss (MFL), based on the focal loss to deal with imbalanced data. Experimental results on EMOTIC dataset show that MFL with the Huber loss gave better results than any other combination and outperformed the current state of art on the less frequent labels.
No file

Dates and versions

hal-03106102 , version 1 (11-01-2021)

Identifiers

Cite

Ilyes Bendjoudi, Frédéric Vanderhaegen, Denis Hamad, Fadi Dornaika. Multi-label, multi-task CNN approach for context-based emotion recognition. Information Fusion, 2021, 76, pp.422-428. ⟨10.1016/j.inffus.2020.11.007⟩. ⟨hal-03106102⟩
92 View
0 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More