Task Agnostic and Task Specific Self-Supervised Learning from Speech with LeBenchmark - Archive ouverte HAL
Conference Papers Year : 2021

Task Agnostic and Task Specific Self-Supervised Learning from Speech with LeBenchmark

Salima Mdhaffar
Natalia Tomashenko
Yannick Estève
Laurent Besacier

Abstract

Self-Supervised Learning (SSL) has yielded remarkable improvements in many different domains including computer vision, natural language processing and speech processing by leveraging large amounts of unlabeled data. In the specific context of speech, however, and despite promising results, there exists a clear lack of standardization in the evaluation process for comprehensive comparisons of these models. This issue gets even worse with the investigation of SSL approaches for other languages than English. We present LeBenchmark, an open-source and reproducible framework for assessing SSL from French speech data. It includes a documented, large-scale and heterogeneous corpora, seven pre-trained SSL wav2vec 2.0 models shared with the community, and a clear evaluation protocol made of four downstream tasks along with their scoring scripts: automatic speech recognition, spoken language understanding, automatic speech translation and automatic emotion recognition. For the first time, SSL models are analyzed and compared on the latter domains both from a task-agnostic (i.e. frozen) and task-specific (i.e. fine-tuned w.r.t the downstream task) perspectives. We report state-of-the-art performance on most considered French tasks and provide a readable evaluation set-up for the development of future SSL models for speech processing.
Fichier principal
Vignette du fichier
FLOWBERT_NEURIPS2021.pdf (393.87 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03407172 , version 1 (07-02-2022)

Identifiers

  • HAL Id : hal-03407172 , version 1

Cite

Solène Evain, Manh Ha Nguyen, Hang Le, Marcely Zanon Boito, Salima Mdhaffar, et al.. Task Agnostic and Task Specific Self-Supervised Learning from Speech with LeBenchmark. Thirty-fifth Conference on Neural Information Processing Systems ( NeurIPS 2021), Dec 2021, on-line, United States. ⟨hal-03407172⟩
861 View
619 Download

Share

More