Human vs Automatic Metrics: on the Importance of Correlation Design - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2020

Human vs Automatic Metrics: on the Importance of Correlation Design

Résumé

This paper discusses two existing approaches to the correlation analysis between automatic evaluation metrics and human scores in the area of natural language generation. Our experiments show that depending on the usage of a system- or sentence-level correlation analysis, correlation results between automatic scores and human judgments are inconsistent.

Dates et versions

hal-02459994 , version 1 (29-01-2020)

Identifiants

Citer

Anastasia Shimorina. Human vs Automatic Metrics: on the Importance of Correlation Design. 2020. ⟨hal-02459994⟩
21 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More