ParBLEU: Augmenting Metrics with Automatic Paraphrases for the WMT'20 Metrics Shared Task - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2020

ParBLEU: Augmenting Metrics with Automatic Paraphrases for the WMT'20 Metrics Shared Task

Résumé

We describe parBLEU, parCHRF++, and parESIM, which augment baseline metrics with automatically generated paraphrases produced by PRISM (Thompson and Post, 2020a), a multilingual neural machine translation system. We build on recent work studying how to improve BLEU by using diverse automatically paraphrased references (Bawden et al., 2020), extending experiments to the multilingual setting for the WMT2020 metrics shared task and for three base metrics. We compare their capacity to exploit up to 100 additional synthetic references. We find that gains are possible when using additional, automatically paraphrased references, although they are not systematic. However, segment-level correlations, particularly into English, are improved for all three metrics and even with higher numbers of paraphrased references.
Fichier principal
Vignette du fichier
Parbleu_system_paper-10.pdf (197.87 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-02981143 , version 1 (27-10-2020)

Identifiants

  • HAL Id : hal-02981143 , version 1

Citer

Rachel Bawden, Biao Zhang, Andre Tättar, Matt Post. ParBLEU: Augmenting Metrics with Automatic Paraphrases for the WMT'20 Metrics Shared Task. 5th Conference on Machine Translation, Nov 2020, Online, Unknown Region. ⟨hal-02981143⟩
112 Consultations
205 Téléchargements

Partager

Gmail Facebook X LinkedIn More