Modular Speech-to-Text Translation for Zero-Shot Cross-Modal Transfer - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Modular Speech-to-Text Translation for Zero-Shot Cross-Modal Transfer

Résumé

Recent research has shown that independently trained encoders and decoders, combined through a shared fixed-size representation, can achieve competitive performance in speech-to-text translation. In this work, we show that this type of approach can be further improved with multilingual training. We observe significant improvements in zero-shot cross-modal speech translation, even outperforming a supervised approach based on XLSR for several languages.
Fichier principal
Vignette du fichier
Modular_speech_to_text_translation_for_zero_shot_cross_modal_transfer.pdf (206.82 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04264023 , version 1 (29-10-2023)

Identifiants

Citer

Paul-Ambroise Duquenne, Holger Schwenk, Benoît Sagot. Modular Speech-to-Text Translation for Zero-Shot Cross-Modal Transfer. INTERSPEECH 2023, Aug 2023, Dublin, Ireland. ⟨10.21437/Interspeech.2023-2484⟩. ⟨hal-04264023⟩
7 Consultations
31 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More