Towards Speech-to-Pictograms Translation - Archive ouverte HAL
Communication Dans Un Congrès Année : 2024

Towards Speech-to-Pictograms Translation

Résumé

The automatic translation of speech into pictogram terms (Speech-to-Pictos) represents a novel NLP task with the potential to enhance communication for individuals with language impairments. Recent research has not explored the adaptation of state-of-the-art methods to this task, despite its significance. In this work, we investigate two approaches: (1) the cascade approach, which combines a speech recognition system with a machine translation system, and (2) the end-to-end approach, which tailors a speech translation system. We compare state-of-the-art architectures trained on an aligned speech-topictogram dataset, specially created and released for this study. We conduct an in-depth automatic and human evaluation to analyze their behavior on pictogram translation. The results highlight the cascade approach's ability to generate relevant translations from everyday read speech, while the end-to-end approach achieves competitive results with challenging acoustic data.
Fichier principal
Vignette du fichier
Interspeech_2024_Propicto.pdf (317.07 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04687483 , version 1 (04-09-2024)

Licence

Identifiants

Citer

Cécile Macaire, Chloé Dion, Didier Schwab, Benjamin Lecouteux, Emmanuelle Esperança-Rodier. Towards Speech-to-Pictograms Translation. Interspeech 2024, Sep 2024, Kos / Greece, Greece. pp.857-861, ⟨10.21437/Interspeech.2024-490⟩. ⟨hal-04687483⟩
8 Consultations
15 Téléchargements

Altmetric

Partager

More