GNEG: Graph-Based Negative Sampling for word2vec - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2018

GNEG: Graph-Based Negative Sampling for word2vec

Résumé

Negative sampling is an important component in word2vec for distributed word representation learning. We hypothesize that taking into account global, corpuslevel information and generating a different noise distribution for each target word better satisfies the requirements of negative examples for each training word than the original frequency-based distribution. In this purpose we pre-compute word cooccurrence statistics from the corpus and apply to it network algorithms such as random walk. We test this hypothesis through a set of experiments whose results show that our approach boosts the word analogy task by about 5\% and improves the performance on word similarity tasks by about 1\% compared to the skip-gram negative sampling baseline.
Fichier non déposé

Dates et versions

hal-01899825 , version 1 (19-10-2018)

Identifiants

  • HAL Id : hal-01899825 , version 1

Citer

Zheng Zhang, Pierre Zweigenbaum. GNEG: Graph-Based Negative Sampling for word2vec. Annual Meeting of the Association for Computational Linguistics, Jul 2018, Melbourne, Australia. ⟨hal-01899825⟩
46 Consultations
0 Téléchargements

Partager

Gmail Facebook X LinkedIn More