Cross-Lingual Contextual Word Embeddings Mapping with Multi-Sense Words in Mind - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2019

Cross-Lingual Contextual Word Embeddings Mapping with Multi-Sense Words in Mind

Zheng Zhang
  • Fonction : Auteur
  • PersonId : 1034325
Ruiqing Yin
  • Fonction : Auteur
  • PersonId : 1034326
Jun Zhu
  • Fonction : Auteur

Résumé

Recent work in cross-lingual contextual word embedding learning cannot handle multi-sense words well. In this work, we explore the characteristics of contextual word embeddings and show the link between contextual word embeddings and word senses. We propose two improving solutions by considering contextual multi-sense word embeddings as noise (removal) and by generating cluster level average anchor embeddings for contextual multi-sense word embeddings (replacement). Experiments show that our solutions can improve the supervised contextual word embeddings alignment for multi-sense words in a microscopic perspective without hurting the macroscopic performance on the bilingual lexicon induction task. For unsupervised alignment, our methods significantly improve the performance on the bilingual lexicon induction task for more than 10 points.
Fichier principal
Vignette du fichier
1909.08681.pdf (3.76 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03100840 , version 1 (06-01-2021)

Identifiants

Citer

Zheng Zhang, Ruiqing Yin, Jun Zhu, Pierre Zweigenbaum. Cross-Lingual Contextual Word Embeddings Mapping with Multi-Sense Words in Mind. 2019. ⟨hal-03100840⟩
55 Consultations
58 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More