Joint speech and overlap detection: a benchmark over multiple audio setup and speech domains - Archive ouverte HAL Accéder directement au contenu
Rapport Année : 2023

Joint speech and overlap detection: a benchmark over multiple audio setup and speech domains

Résumé

Voice activity and overlapped speech detection (respectively VAD and OSD) are key pre-processing tasks for speaker diarization. The final segmentation performance highly relies on the robustness of these sub-tasks. Recent studies have shown VAD and OSD can be trained jointly using a multi-class classification model. However, these works are often restricted to a specific speech domain, lacking information about the generalization capacities of the systems. This paper proposes a complete and new benchmark of different VAD and OSD models, on multiple audio setups (single/multi-channel) and speech domains (e.g. media, meeting...). Our 2/3-class systems, which combine a Temporal Convolutional Network with speech representations adapted to the setup, outperform state-of-the-art results. We show that the joint training of these two tasks offers similar performances in terms of F1-score to two dedicated VAD and OSD systems while reducing the training cost. This unique architecture can also be used for single and multichannel speech processing.
Fichier principal
Vignette du fichier
Speech_activity_overlap_benchmark.pdf (488.28 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04133268 , version 1 (24-07-2023)

Identifiants

Citer

Martin Lebourdais, Théo Mariotte, Marie Tahon, Anthony Larcher, Antoine Laurent, et al.. Joint speech and overlap detection: a benchmark over multiple audio setup and speech domains. Le Mans Université. 2023. ⟨hal-04133268⟩
91 Consultations
47 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More