The Impact of Word Splitting on the Semantic Content of Contextualized Word Representations - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Transactions of the Association for Computational Linguistics Année : 2024

The Impact of Word Splitting on the Semantic Content of Contextualized Word Representations

Résumé

When deriving contextualized word representations from language models, a decision needs to be made on how to obtain one for out-of-vocabulary (OOV) words that are segmented into subwords. What is the best way to represent these words with a single vector, and are these representations of worse quality than those of in-vocabulary words? We carry out an intrinsic evaluation of embeddings from different models on semantic similarity tasks involving OOV words. Our analysis reveals, among other interesting findings, that the quality of representations of words that are split is often, but not always, worse than that of the embeddings of known words. Their similarity values, however, must be interpreted with caution.
Fichier principal
Vignette du fichier
tacl_a_00647.pdf (570.32 Ko) Télécharger le fichier
Origine Fichiers éditeurs autorisés sur une archive ouverte

Dates et versions

hal-04593480 , version 1 (30-05-2024)

Identifiants

Citer

Aina Garí Soler, Matthieu Labeau, Chloé Clavel. The Impact of Word Splitting on the Semantic Content of Contextualized Word Representations. Transactions of the Association for Computational Linguistics, 2024, 12, pp.299-320. ⟨10.1162/tacl_a_00647⟩. ⟨hal-04593480⟩
17 Consultations
9 Téléchargements

Altmetric

Partager

Gmail Mastodon Facebook X LinkedIn More