Communication Dans Un Congrès Année : 2024

Using Whisper to Investigate Learner Pronunciations of English: comparing LLM transcriptions with human perception of VOT

Résumé

This talk will show that Whisper large Language Model (LLM) is a tool for providing automatic pronunciation feedback and phonetic diagnoses to L2 learners using Automatic Speech Recognition (ASR) transcriptions of audio files and that the tiny model is an accurate representation of human interpretation as evidenced by the transcriptions of bilabial plosives according to VOT variability.

Whisper is an audio Pretrained Large Language Model (PLLM) that can be used for both transcription and translation tasks (Radford et al., 2023). There are seven main models within Whisper that provide varying transcriptions for the same audio input. The difference between these models is the number of parameters as well as how much data each model has been trained on. These models also include multilingual models that perform a language detection task before the transcription task (tiny, small, base, medium, large, large-v1, and large-v2) as well as native .en models (base.en, medium.en, small.en, and tiny.en) that assume the input is in English. The main goal of this research is to investigate if the ASR output of Whisper is consistent with the human judgments produced by Native English speakers and L2 learners. More specifically, we wish to determine if the tiny models can be trusted to produce transcriptions consistent with human interpretations.

Cette présentation analyse la production d'apprenants francophones de l'anglais (L2) avec Whisper et étudie la transcription automatique produite par les différents modèles lorsque le VOT varie.

Fichier principal
Vignette du fichier
FullertonBallier2024.pdf (153.03 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Licence

Dates et versions

hal-04911801 , version 1 (25-01-2025)

Licence

Identifiants

  • HAL Id : hal-04911801 , version 1

Citer

Tori Fullerton, Nicolas Ballier. Using Whisper to Investigate Learner Pronunciations of English: comparing LLM transcriptions with human perception of VOT. 21e colloque d'anglais oral de Villetaneuse, ALOES; Université Sorbonne Paris Nord (Pléiade – UR 7338), Mar 2024, Villetaneuse (Université Paris 13), France. ⟨hal-04911801⟩
117 Consultations
194 Téléchargements

Partager

  • More