Predicting Word Embeddings Variability - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2018

Predicting Word Embeddings Variability

Résumé

Neural word embeddings models (such as those built with word2vec) are known to have stability problems: when retraining a model with the exact same hyperparameters, words neighborhoods may change. We propose a method to estimate such variation, based on the overlap of neighbors of a given word in two models trained with identical hyperparam-eters. We show that this inherent variation is not negligible, and that it does not affect every word in the same way. We examine the influence of several features that are intrinsic to a word, corpus or embedding model and provide a methodology that can predict the variability (and as such, reliability) of a word representation in a semantic vector space.
Fichier principal
Vignette du fichier
document.pdf (152.76 Ko) Télécharger le fichier
Origine : Fichiers éditeurs autorisés sur une archive ouverte
Loading...

Dates et versions

hal-01806467 , version 1 (02-06-2018)

Identifiants

  • HAL Id : hal-01806467 , version 1

Citer

Bénédicte Pierrejean, Ludovic Tanguy. Predicting Word Embeddings Variability. The seventh Joint Conference on Lexical and Computational Semantics, Jun 2018, New Orleans, United States. pp.154 - 159. ⟨hal-01806467⟩
193 Consultations
211 Téléchargements

Partager

Gmail Facebook X LinkedIn More