Analyzing Speech Unit Selection for Textless Speech-to-Speech Translation - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2024

Analyzing Speech Unit Selection for Textless Speech-to-Speech Translation

Résumé

Recent advancements in textless speech-to-speech translation systems have been driven by the adoption of self-supervised learning techniques. Although most state-of-the-art systems adopt a similar architecture to transform source language speech into sequences of discrete representations in the target language, the criteria for selecting these target speech units remains an open question. This work explores the selection process through a study of downstream tasks such as automatic speech recognition, speech synthesis, speaker recognition, and emotion recognition. Interestingly, our findings reveal a discrepancy in the optimization of discrete speech units: units that perform well in resynthesis performance do not necessarily correlate with those that enhance translation efficacy. This discrepancy underscores the nuanced complexity of target feature selection and its impact on the overall performance of speech-to-speech translation systems.
Fichier principal
Vignette du fichier
template.pdf (159.68 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04628397 , version 1 (07-07-2024)

Identifiants

  • HAL Id : hal-04628397 , version 1

Citer

Jarod Duret, Yannick Estève, Titouan Parcollet. Analyzing Speech Unit Selection for Textless Speech-to-Speech Translation. 2024. ⟨hal-04628397⟩

Collections

UNIV-AVIGNON LIA
0 Consultations
0 Téléchargements

Partager

Gmail Mastodon Facebook X LinkedIn More