Better Evaluation of ASR in Speech Translation Context Using Word Embeddings - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2016

Better Evaluation of ASR in Speech Translation Context Using Word Embeddings

Résumé

This paper investigates the evaluation of ASR in spoken language translation context. More precisely, we propose a simple extension of WER metric in order to penalize differently substitution errors according to their context using word embeddings. For instance, the proposed metric should catch near matches (mainly morphological variants) and penalize less this kind of error which has a more limited impact on translation performance. Our experiments show that the correlation of the new proposed metric with SLT performance is better than the one of WER. Oracle experiments are also conducted and show the ability of our metric to find better hypotheses (to be translated) in the ASR N-best. Finally, a preliminary experiment where ASR tuning is based on our new metric shows encouraging results. For reproductible experiments, the code allowing to call our modified WER and the corpora used are made available to the research community.
Fichier principal
Vignette du fichier
metrics_correlation_asr-smt.pdf (150.05 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01350102 , version 1 (29-07-2016)

Identifiants

  • HAL Id : hal-01350102 , version 1

Citer

Ngoc-Tien Le, Christophe Servan, Benjamin Lecouteux, Laurent Besacier. Better Evaluation of ASR in Speech Translation Context Using Word Embeddings. Interspeech 2016, Sep 2016, San-Francisco, United States. ⟨hal-01350102⟩
430 Consultations
1130 Téléchargements

Partager

Gmail Facebook X LinkedIn More