Contrasting distinct structured views to learn sentence embeddings - Archive ouverte HAL Access content directly
Conference Papers Year : 2021

Contrasting distinct structured views to learn sentence embeddings

Antoine Simoulin
Benoît Crabbé
  • Function : Author
  • PersonId : 1098477


We propose a self-supervised method that builds sentence embeddings from the combination of diverse explicit syntactic structures of a sentence. We assume structure is crucial to building consistent representations as we expect sentence meaning to be a function of both syntax and semantic aspects. In this perspective, we hypothesize that some linguistic representations might be better adapted given the considered task or sentence. We, therefore, propose to learn individual representation functions for different syntactic frameworks jointly. Again, by hypothesis, all such functions should encode similar semantic information differently and consequently, be complementary for building better sentential semantic embeddings. To assess such hypothesis, we propose an original contrastive multi-view framework that induces an explicit interaction between models during the training phase. We make experiments combining various structures such as dependency, constituency, or sequential schemes. Our results outperform comparable methods on several tasks from standard sentence embedding benchmarks.
Fichier principal
Vignette du fichier
2021.eacl-srw.11.pdf (485.84 Ko) Télécharger le fichier
Origin : Publisher files allowed on an open archive

Dates and versions

hal-03601428 , version 1 (08-03-2022)


  • HAL Id : hal-03601428 , version 1


Antoine Simoulin, Benoît Crabbé. Contrasting distinct structured views to learn sentence embeddings. European Chapter of the Association of Computational Linguistics (student), 2021, Kyiv, Ukraine. ⟨hal-03601428⟩
27 View
21 Download


Gmail Facebook X LinkedIn More