Communication Dans Un Congrès Année : 2025

Mixture of LoRA Experts for Low-Resourced Multi-Accent Automatic Speech Recognition

Résumé

We aim to improve the robustness of Automatic Speech Recognition (ASR) systems against non-native speech, particularly in low-resourced multi-accent settings. We introduce Mixture of Accent-Specific LoRAs (MAS-LoRA), a finetuning method that leverages a mixture of Low-Rank Adaptation (LoRA) experts, each specialized in a specific accent. This method can be used when the accent is known or unknown at inference time, without the need to fine-tune the model again. Our experiments, conducted using Whisper on the L2-ARCTIC corpus, demonstrate significant improvements in Word Error Rate compared to regular LoRA and full fine-tuning when the accent is unknown. When the accent is known, the results further improve. Furthermore, MAS-LoRA shows less catastrophic forgetting than the other fine-tuning methods. To the best of our knowledge, this is the first use of a mixture of LoRA experts for non-native multi-accent ASR.

Fichier principal
Vignette du fichier
camready_maslora-1.pdf (793.25 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Licence

Dates et versions

hal-05088547 , version 1 (28-05-2025)

Licence

Identifiants

Citer

Raphaël Bagat, Irina Illina, Emmanuel Vincent. Mixture of LoRA Experts for Low-Resourced Multi-Accent Automatic Speech Recognition. 26th Interspeech Conference (Interspeech 2025), Aug 2025, Rotterdam, Netherlands. ⟨hal-05088547⟩
539 Consultations
314 Téléchargements

Altmetric

Partager

  • More