Some Preliminary Results on Analogies Between Sentences Using Contextual and Non-Contextual Embeddings - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Some Preliminary Results on Analogies Between Sentences Using Contextual and Non-Contextual Embeddings

Résumé

Analogies have been characterized as fundamental to abstraction, concept formation, and perception, and are traditionally expressed as quadruplets in the form of proportional analogies a : b :: c : d read "a is to b as c is to d". While Natural Language Processing (NLP) has primarily focused on word analogies and SAT problems, recent research has started exploring analogies between sentences and even documents. In this paper we explore the potential of identifying analogies between pairs of sentences via the identification of common latent relations between them. We exploit three different datasets generating pairs of sentences which can either share the same latent relation—forming thus an analogy—or not. We encode phrases into a higher dimensional vector space using embeddings from GloVe, BERT, and RoBERTa which we then feed to both a Multi Layer Perceptron (MLP) and a Convolutional Neural Network (CNN). Results show that architectures using contextual embeddings as inputs outperform those based on static embeddings
Fichier principal
Vignette du fichier
paper4.pdf (6 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04394281 , version 1 (15-01-2024)

Licence

Paternité

Identifiants

  • HAL Id : hal-04394281 , version 1

Citer

Stergos Afantenos, Thomas Barbero. Some Preliminary Results on Analogies Between Sentences Using Contextual and Non-Contextual Embeddings. 2nd Workshop on the Interactions between Analogical Reasoning and Machine Learning 2023 (IARML 2023), Aug 2023, Macao SAR, China. pp.34--45. ⟨hal-04394281⟩
72 Consultations
15 Téléchargements

Partager

Gmail Facebook X LinkedIn More