A Paradigm for Interpreting Metrics and Measuring Error Severity in Automatic Speech Recognition - Archive ouverte HAL
Communication Dans Un Congrès Année : 2024

A Paradigm for Interpreting Metrics and Measuring Error Severity in Automatic Speech Recognition

Résumé

The evaluation of automatic speech transcriptions relies heavily on metrics such as Word Error Rate (WER) and Character Error Rate (CER). However, these metrics have faced criticism for their limited correlation with human perception and their inability to capture linguistic and semantic nuances accurately. Despite the introduction of metric-based embeddings to approximate human perception, their interpretability remains challenging compared to traditional metrics. In this article, we introduce a novel paradigm aimed at addressing these limitations. Our approach integrates a chosen metric to derive Minimum Edit Distance (minED), which serves as an indicator of the rate of serious errors in automatic speech transcriptions. Unlike conventional metrics, minED offers a more nuanced understanding of errors, accounting for both linguistic complexities and human perception. Furthermore, our paradigm facilitates the measurement of error severity from both intrinsic and extrinsic perspectives.
Fichier principal
Vignette du fichier
TSD_2024___minED.pdf (453.39 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04615039 , version 1 (17-06-2024)
hal-04615039 , version 2 (23-06-2024)

Identifiants

  • HAL Id : hal-04615039 , version 2

Citer

Thibault Bañeras-Roux, Mickael Rouvier, Jane Wottawa, Richard Dufour. A Paradigm for Interpreting Metrics and Measuring Error Severity in Automatic Speech Recognition. Text, Speech and Dialogue, 2024, Brno, Czech Republic. ⟨hal-04615039v2⟩
226 Consultations
66 Téléchargements

Partager

More