Some Preliminary Results on Analogies Between Sentences Using Contextual and Non-Contextual Embeddings - Archive ouverte HAL Access content directly
Conference Papers Year : 2023

Some Preliminary Results on Analogies Between Sentences Using Contextual and Non-Contextual Embeddings

Abstract

Analogies have been characterized as fundamental to abstraction, concept formation, and perception, and are traditionally expressed as quadruplets in the form of proportional analogies a : b :: c : d read "a is to b as c is to d". While Natural Language Processing (NLP) has primarily focused on word analogies and SAT problems, recent research has started exploring analogies between sentences and even documents. In this paper we explore the potential of identifying analogies between pairs of sentences via the identification of common latent relations between them. We exploit three different datasets generating pairs of sentences which can either share the same latent relation—forming thus an analogy—or not. We encode phrases into a higher dimensional vector space using embeddings from GloVe, BERT, and RoBERTa which we then feed to both a Multi Layer Perceptron (MLP) and a Convolutional Neural Network (CNN). Results show that architectures using contextual embeddings as inputs outperform those based on static embeddings
Fichier principal
Vignette du fichier
paper4.pdf (6 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-04394281 , version 1 (15-01-2024)

Licence

Attribution

Identifiers

  • HAL Id : hal-04394281 , version 1

Cite

Stergos Afantenos, Thomas Barbero. Some Preliminary Results on Analogies Between Sentences Using Contextual and Non-Contextual Embeddings. 2nd Workshop on the Interactions between Analogical Reasoning and Machine Learning 2023 (IARML 2023), Aug 2023, Macao SAR, China. pp.34--45. ⟨hal-04394281⟩
72 View
15 Download

Share

Gmail Facebook X LinkedIn More