Using Whisper LLM for Automatic Phonetic Diagnosis of L2 Speech: A Case Study with French Learners of English - Archive ouverte HAL
Communication Dans Un Congrès Année : 2023

Using Whisper LLM for Automatic Phonetic Diagnosis of L2 Speech: A Case Study with French Learners of English

Résumé

This paper reports on a pilot study to use Whisper’s large language model (LLM) as a tool for potential representation of segmental (phone) pronunciation errors. We compared the perfor- mance of the transcription outputs for the various models developed by the automatic speech recognition (ASR) system Whisper (Radford et al., 2022) ranging from 39 to 1,550 million parameters. We investigated 38 recordings of two paragraphs from Conrad’s Typhoon. The whisper transcriptions were compared to the original text that was read by these second-year French undergraduates. We used WER (Word Error Rate) and Levenshtein distance to assess the various graphic representations of Conrad’s reference text. We show how the differences can be transformed into operationalised feed- back for learners. We used expert phonetic knowledge to check the plausibility of the pho- netic interpretation with the signal (in particular the recall of H dropping produced by French learners). Our findings suggest that the tran- scriptions produced by the medium model con- verge with what a native speaker understands and that the tiny model produces alternate transcriptions that are plausible candidates for learner errors.
Fichier principal
Vignette du fichier
WhisperLLM.pdf (333.13 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04547597 , version 1 (15-04-2024)

Licence

Identifiants

  • HAL Id : hal-04547597 , version 1

Citer

Nicolas Ballier, Adrien Méli, Maelle Amand, Jean-Baptiste Yunès. Using Whisper LLM for Automatic Phonetic Diagnosis of L2 Speech: A Case Study with French Learners of English. 6th International Conference on Natural Language and Speech Processing (ICNLSP 2023), Mourad Abbas, Abed Alhakim Freihat, Dec 2023, Trento (Italy), Italy. ⟨hal-04547597⟩
366 Consultations
279 Téléchargements

Partager

More