Speech Self-Supervised Representation Benchmarking: Are We Doing it Right? - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Speech Self-Supervised Representation Benchmarking: Are We Doing it Right?

Résumé

Self-supervised learning (SSL) has recently allowed leveraging large datasets of unlabeled speech signals to reach impressive performance on speech tasks using only small amounts of annotated data. The high number of proposed approaches fostered the need and rise of extended benchmarks that evaluate their performance on a set of downstream tasks exploring various aspects of the speech signal. However, and while the number of considered tasks has been growing, most rely upon a single decoding architecture that maps the frozen SSL representations to the downstream labels. This work investigates the robustness of such benchmarking results to changes in the decoder architecture. Interestingly, it appears that varying the architecture of the downstream decoder leads to significant variations in the leaderboards of most tasks. Concerningly, our study reveals that benchmarking using limited decoders may cause a counterproductive increase in the sizes of the developed SSL models.
Fichier principal
Vignette du fichier
zaiem23b_interspeech.pdf (240.36 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04216175 , version 1 (23-09-2023)

Identifiants

Citer

Salah Zaiem, Youcef Kemiche, Titouan Parcollet, Slim Essid, Mirco Ravanelli. Speech Self-Supervised Representation Benchmarking: Are We Doing it Right?. INTERSPEECH 2023, Aug 2023, Dublin, Ireland. pp.2873-2877, ⟨10.21437/Interspeech.2023-1087⟩. ⟨hal-04216175⟩
16 Consultations
68 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More