Sequence graphs realizations and ambiguity in language models - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2020

Sequence graphs realizations and ambiguity in language models

Résumé

Several natural language models rely on an assumption modeling each word context as a bag of words. We study the combinatorial implications of such assumption for the corresponding word or sentences representations. In particular , we present theoretical results concerning the family of sequence graphs, for which realizations yield equivalent representations given this assumption. Several combinatorial problems are presented, depending on three levels of generalisation (window size, graph orientation, and weights), and whether some of these are NP-complete is left opened. Based on these results, we also establish different algorithms, including a dynamic programming formulation, to count and explicit the different realizations of a sequence graph. This allows us to show that the bag of words assumption can induce an important number of sentences to have the same representations, even for relatively short context window sizes.
Fichier principal
Vignette du fichier
Sequences_graphs_language_models_ISCO_20_sk_yp.pdf (337.25 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-02495333 , version 1 (01-03-2020)
hal-02495333 , version 2 (04-03-2020)
hal-02495333 , version 3 (13-01-2021)
hal-02495333 , version 4 (18-05-2023)

Identifiants

  • HAL Id : hal-02495333 , version 1

Citer

Sammy Khalife, Yann Ponty. Sequence graphs realizations and ambiguity in language models. 2020. ⟨hal-02495333v1⟩
161 Consultations
217 Téléchargements

Partager

Gmail Facebook X LinkedIn More