Brouhaha: multi-task training for voice activity detection, speech-to-noise ratio, and C50 room acoustics estimation - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Brouhaha: multi-task training for voice activity detection, speech-to-noise ratio, and C50 room acoustics estimation

Résumé

Most automatic speech processing systems register degraded performance when applied to noisy or reverberant speech. But how can one tell whether speech is noisy or reverberant? We propose Brouhaha, a neural network jointly trained to extract speech/non-speech segments, speech-to-noise ratios, and C50room acoustics from single-channel recordings. Brouhaha is trained using a data-driven approach in which noisy and reverberant audio segments are synthesized. We first evaluate its performance and demonstrate that the proposed multi-task regime is beneficial. We then present two scenarios illustrating how Brouhaha can be used on naturally noisy and reverberant data: 1) to investigate the errors made by a speaker diarization model (pyannote.audio); and 2) to assess the reliability of an automatic speech recognition model (Whisper from OpenAI). Both our pipeline and a pretrained model are open source and shared with the speech community.
Fichier principal
Vignette du fichier
_ASRU__Brouhaha-1.pdf (565.76 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04247647 , version 1 (20-10-2023)

Identifiants

Citer

Marvin Lavechin, Marianne Métais, Hadrien Titeux, Alodie Boissonnet, Jade Copet, et al.. Brouhaha: multi-task training for voice activity detection, speech-to-noise ratio, and C50 room acoustics estimation. IEEE Automatic Speech Recognition and Understanding (ASRU 2023 ), IEEE, Dec 2023, Taipei, Taiwan. pp.1--7. ⟨hal-04247647⟩
149 Consultations
62 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More