Learning Word Importance with the Neural Bag-of-Words Model - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2016

Learning Word Importance with the Neural Bag-of-Words Model

Résumé

The Neural Bag-of-Words (NBOW) model performs classification with an average of the input word vectors and achieves an impressive performance. While the NBOW model learns word vectors targeted for the classification task it does not explicitly model which words are important for given task. In this paper we propose an improved NBOW model with this ability to learn task specific word importance weights. The word importance weights are learned by introducing a new weighted sum composition of the word vectors. With experiments on standard topic and sentiment classification tasks, we show that (a) our proposed model learns meaningful word importance for a given task (b) our model gives best accuracies among the BOW approaches. We also show that the learned word importance weights are comparable to tf-idf based word weights when used as features in a BOWSVM classifier.
Fichier principal
Vignette du fichier
repl4nlp_draft21Jun16.pdf (573.39 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01331720 , version 1 (20-10-2016)

Identifiants

  • HAL Id : hal-01331720 , version 1

Citer

Imran Sheikh, Irina Illina, Dominique Fohr, Georges Linares. Learning Word Importance with the Neural Bag-of-Words Model. ACL, Representation Learning for NLP (Repl4NLP) workshop, Aug 2016, Berlin, Germany. ⟨hal-01331720⟩
926 Consultations
505 Téléchargements

Partager

Gmail Facebook X LinkedIn More