On the (In)Efficiency of Acoustic Feature Extractors for Self-Supervised Speech Representation Learning - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

On the (In)Efficiency of Acoustic Feature Extractors for Self-Supervised Speech Representation Learning

Résumé

Speech representations learned with self-supervised learning (SSL) have the potential to significantly improve the performance of a number of audio applications, especially when availability of labeled data from the deployment domain is limited. Despite their successes, SSL training methods are compute-and memory-heavy, and require large investments in computing infrastructure, thus putting it out of the reach of most institutions. Therefore, building efficient model architectures is essential for the wide-scale adoption of SSL in speech technologies. CNN-based Acoustic Feature Extractors (AFE), which are widely used as encoders of acoustic waveforms, remain one of the main efficiency bottlenecks. This work proposes replacing CNN-based AFEs with more efficient ones and demonstrates that SSL pre-training time and memory consumption can be reduced by a factor of two to three over existing methods while preserving performances in speech-, command-, and speakerrecognition tasks.
Fichier principal
Vignette du fichier
_IS2023__On_the__In_Efficiency_of_Acoustic_Features_Extractors_for_Self_Supervised_Speech_Representation_Learning-2.pdf (135.56 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04116371 , version 1 (03-06-2023)

Identifiants

  • HAL Id : hal-04116371 , version 1

Citer

Titouan Parcollet, Shucong Zhang, Rogier van Dalen, Alberto Gil C. P. Ramos, Sourav Bhattacharya. On the (In)Efficiency of Acoustic Feature Extractors for Self-Supervised Speech Representation Learning. Interspeech 2023, Aug 2023, Dublin, France. ⟨hal-04116371⟩
536 Consultations
413 Téléchargements

Partager

Gmail Facebook X LinkedIn More