Overlapped speech and gender detection with WavLM pre-trained features - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

Overlapped speech and gender detection with WavLM pre-trained features

Résumé

This article focuses on overlapped speech and gender detection in order to study interactions between women and men in French audiovisual media (Gender Equality Monitoring project). In this application context, we need to automatically segment the speech signal according to speakers gender, and to identify when at least two speakers speak at the same time. We propose to use WavLM model which has the advantage of being pre-trained on a huge amount of speech data, to build an overlapped speech detection (OSD) and a gender detection (GD) systems. In this study, we use two different corpora. The DIHARD III corpus which is well adapted for the OSD task but lack gender information. The ALLIES corpus fits with the project application context. Our best OSD system is a Temporal Convolutional Network (TCN) with WavLM pre-trained features as input, which reaches a new state-of-the-art F1-score performance on DIHARD. A neural GD is trained with WavLM inputs on a gender balanced subset of the French broadcast news ALLIES data, and obtains an accuracy of 97.9%. This work opens new perspectives for human science researchers regarding the differences of representation between women and men in French media.
Fichier principal
Vignette du fichier
Interspeech2022_v5.pdf (183.36 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03770634 , version 1 (06-09-2022)

Identifiants

  • HAL Id : hal-03770634 , version 1

Citer

Martin Lebourdais, Marie Tahon, Antoine Laurent, Sylvain Meignier. Overlapped speech and gender detection with WavLM pre-trained features. Interspeech, Sep 2022, Incheon, South Korea. ⟨hal-03770634⟩
169 Consultations
198 Téléchargements

Partager

Gmail Facebook X LinkedIn More