Strength Modelling for Real-World Automatic Continuous Affect Recognition from Audiovisual Signals - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Image and Vision Computing Année : 2017

Strength Modelling for Real-World Automatic Continuous Affect Recognition from Audiovisual Signals

Résumé

Automatic continuous affect recognition from audiovisual cues is arguably one of the most active research areas in machine learning. In addressing this regression problem, the advantages of the models, such as the global-optimisation capability of Support Vector Machine for Regression and the context-sensitive capability of memory-enhanced neural networks, have been frequently explored, but in an isolated way. Motivated to leverage the individual advantages of these techniques, this paper proposes and explores a novel framework, Strength Modelling, where two models are concatenated in a hierarchical framework. In doing this, the strength information of the first model, as represented by its predictions, is joined with the original features, and this expanded feature space is then utilised as the input by the successive model. A major advantage of Strength Modelling, besides its ability to hierarchically explore the strength of different machine learning algorithms, is that it can work together with the conventional feature-and decision-level fusion strategies for multimodal affect recognition. To highlight the effectiveness and robustness of the proposed approach, extensive experiments have been carried out on two time-and value-continuous spontaneous emotion databases (RECOLA and SEMAINE) using audio and video signals. The experimental results indicate that employing Strength Modelling can deliver a significant performance improvement for both arousal and valence in the unimodal and bimodal settings. The results further show that the proposed systems is competitive or outperform the other state-of-the-art approaches, but being with a simple implementation.
Fichier principal
Vignette du fichier
Han16-STF.pdf (1.34 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01486186 , version 1 (09-03-2017)

Licence

Domaine public

Identifiants

Citer

Jing Han, Zixing Zhang, Nicholas Cummins, Fabien Ringeval, Björn Schuller. Strength Modelling for Real-World Automatic Continuous Affect Recognition from Audiovisual Signals. Image and Vision Computing, 2017, 65, pp.76-86. ⟨10.1016/j.imavis.2016.11.020⟩. ⟨hal-01486186⟩
204 Consultations
459 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More