On "Human Parity" and "Super Human Performance" in Machine Translation Evaluation - Archive ouverte HAL Access content directly
Conference Papers Year : 2022

On "Human Parity" and "Super Human Performance" in Machine Translation Evaluation

Abstract

In this paper, we reassess claims of human parity and super human performance in machine translation. These terms have already been discussed, as well as the evaluation protocols used to achieved these conclusions (human-parity is achieved i) only for a very reduced number of languages, ii) on very specific types of documents and iii) with very literal translations). However, we think it is necessary to consider these questions again. We show that the terms used are themselves problematic, and that human translation involves much more than what is embedded in automatic systems. We also discuss ethical issues related to the way results are presented and advertised. Finally, we claim that a better assessment of human capacities should be put forward and that the goal of replacing humans by machines is not a desirable one.
Fichier principal
Vignette du fichier
LREC2022_poibeau.pdf (104 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03738720 , version 1 (26-07-2022)

Identifiers

  • HAL Id : hal-03738720 , version 1

Cite

Thierry Poibeau. On "Human Parity" and "Super Human Performance" in Machine Translation Evaluation. Language Resource and Evaluation Conference, Jun 2022, Marseille, France. ⟨hal-03738720⟩
189 View
266 Download

Share

Gmail Mastodon Facebook X LinkedIn More