Improvement of emotion recognition from facial images using deep learning and early stopping cross validation - Archive ouverte HAL
Article Dans Une Revue Multimedia Tools and Applications Année : 2022

Improvement of emotion recognition from facial images using deep learning and early stopping cross validation

Résumé

Inthis paper, we present a new approach for emotion recognition from facial images. The proposed method is based on the association of a pretrained convolutional neural network (CNN) model (VGG16, ResNet50) with a multilayer perceptron (MLP) classifier. The pretrained CNN model is used as a feature extractor. For this purpose, we adapt the original architecture by adding a global average pooling layer (GAP) without any fine tuning of the network parameters. In order to avoid overfitting for the MLP classifier, we introduce the early stopping criterion. It is proved that the aforementioned elements contribute in improving the performance of our approach in terms of generalization ability. The procedure for emotion recognition from facial images is applied on the CK+ (extended Cohen-Kanad), JAFFE (Japanese Female Facial Expression) and KDEF (Karolinska Directed Emotional Faces) databases. The k-fold cross validation procedure is used for accuracy estimation. The experimental results show the effectiveness of our facial emotion recognition (FER) approach compared to the existing methods yielding to recognition rates of 100%, 96.40% and 98.78% for the CK+, JAFFE and KDEF databases, respectively. On the other hand, further improvement of our recognition performance is obtained for images from the JAFFE database by performing a data augmentation during the training phase. This allows to achieve an accuracy of 100% for this database. Four other metrics, namely the F1-score, positive and negative likelihood ratios and Mathews correlation coefficient, confirm as well the classification results obtained from accuracy.
Fichier non déposé

Dates et versions

hal-03635826 , version 1 (08-04-2022)

Identifiants

Citer

Mohamed Bentoumi, Mohamed Daoud, Mohamed Benaouali, Abdelmalik Taleb-Ahmed. Improvement of emotion recognition from facial images using deep learning and early stopping cross validation. Multimedia Tools and Applications, 2022, 81, pp.29887-29917. ⟨10.1007/s11042-022-12058-0⟩. ⟨hal-03635826⟩
42 Consultations
0 Téléchargements

Altmetric

Partager

More