Communication Dans Un Congrès Année : 2018

Predicting Word Embeddings Variability

Résumé

Neural word embeddings models (such as those built with word2vec) are known to have stability problems: when retraining a model with the exact same hyperparameters, words neighborhoods may change. We propose a method to estimate such variation, based on the overlap of neighbors of a given word in two models trained with identical hyperparam-eters. We show that this inherent variation is not negligible, and that it does not affect every word in the same way. We examine the influence of several features that are intrinsic to a word, corpus or embedding model and provide a methodology that can predict the variability (and as such, reliability) of a word representation in a semantic vector space.

Fichier principal
Vignette du fichier
document.pdf (152.76 Ko) Télécharger le fichier
Origine Fichiers éditeurs autorisés sur une archive ouverte
Licence
Loading...

Dates et versions

hal-01806467 , version 1 (02-06-2018)

Licence

Identifiants

  • HAL Id : hal-01806467 , version 1

Citer

Bénédicte Pierrejean, Ludovic Tanguy. Predicting Word Embeddings Variability. The seventh Joint Conference on Lexical and Computational Semantics, Jun 2018, New Orleans, United States. pp.154 - 159. ⟨hal-01806467⟩
319 Consultations
346 Téléchargements

Partager

  • More