Comparing phoneme recognition systems on the detection and diagnosis of reading mistakes for young children's oral reading evaluation - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Comparing phoneme recognition systems on the detection and diagnosis of reading mistakes for young children's oral reading evaluation

Résumé

In the scope of our oral reading exercise for 5-8-year-old children, models need to be able to precisely detect and diagnose reading mistakes, which remains a considerable challenge even for state-of-the-art ASR systems. In this paper, we compare hybrid and end-to-end acoustic models trained for phoneme recognition on young learners' speech. We evaluate them not only with phoneme error rates but through detailed phoneme-level misread detection and diagnostic metrics. We show that a traditional TDNNF-HMM model, despite a high PER, is the best at detecting reading mistakes (F1-score 72.6%), but at the cost of low precision (73.8%) and specificity (74.7%), which is pedagogically critical. A recent Transformer+CTC model, to which we applied our synthetic reading mistakes augmentation method, obtains the highest precision (81.8%) and specificity (86.3%), as well as the highest correct diagnosis rate (70.7%), showing it is the best fit for our application.
Fichier principal
Vignette du fichier
Comparing phoneme recognition systems on the detection and diagnosis of reading mistakes for young childrens oral reading evaluation.pdf (244.17 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04190328 , version 1 (29-08-2023)

Identifiants

Citer

Lucile Gelin, Morgane Daniel, Thomas Pellegrini, Julien Pinquier. Comparing phoneme recognition systems on the detection and diagnosis of reading mistakes for young children's oral reading evaluation. Speech and Language Technologies in Education (SLaTE 2023), Aug 2023, Dublin, Ireland. pp.6--10, ⟨10.21437/slate.2023-2⟩. ⟨hal-04190328⟩
61 Consultations
128 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More