Communication Dans Un Congrès Année : 2011

Large vocabulary SOUL neural network language models

Résumé

This paper presents continuation of research on Structured OUtput Layer Neural Network language models (SOUL NNLM) for automatic speech recognition. As SOUL NNLMs allow estimating probabilities for all in-vocabulary words and not only for those pertaining to a limited shortlist, we investigate its performance on a large-vocabulary task. Significant improvements both in perplexity and word error rate over conventional shortlist-based NNLMs are shown on a challenging Arabic GALE task characterized by a recognition vocabulary of about 300k entries. A new training scheme is proposed for SOUL NNLMs that is based on separate training of the out-of-shortlist part of the output layer. It enables using more data at each iteration of a neural network without any considerable slow-down in training and brings additional improvements in speech recognition performance.

Fichier non déposé

Dates et versions

hal-01960743 , version 1 (19-12-2018)

Identifiants

Citer

Hai Son Le, Ilya Oparin, Abdelkhalek Messaoudi, Alexandre Allauzen, Jean-Luc Gauvain, et al.. Large vocabulary SOUL neural network language models. Annual Conference of the International Speech Communication Association, Jan 2011, Florence, Italy. ⟨10.21437/Interspeech.2011-259⟩. ⟨hal-01960743⟩
122 Consultations
0 Téléchargements

Altmetric

Partager

  • More