On the Robustness of Text Vectorizers - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

On the Robustness of Text Vectorizers

Résumé

A fundamental issue in machine learning is the robustness of the model with respect to changes in the input. In natural language processing, models typically contain a first embedding layer, transforming a sequence of tokens into vector representations. While the robustness with respect to changes of continuous inputs is well-understood, the situation is less clear when considering discrete changes, for instance replacing a word by another in an input sentence. Our work formally proves that popular embedding schemes, such as concatenation, TF-IDF, and Paragraph Vector (a.k.a. doc2vec), exhibit robustness in the H\"older or Lipschitz sense with respect to the Hamming distance. We provide quantitative bounds for these schemes and demonstrate how the constants involved are affected by the length of the document. These findings are exemplified through a series of numerical examples.
Fichier principal
Vignette du fichier
catellier23a.pdf (949.93 Ko) Télécharger le fichier
Origine : Fichiers éditeurs autorisés sur une archive ouverte

Dates et versions

hal-04403681 , version 1 (18-01-2024)

Identifiants

Citer

Rémi Catellier, Samuel Vaiter, Damien Garreau. On the Robustness of Text Vectorizers. ICML 2023 - Fortieth International Conference on Machine Learning, Jul 2023, Honolulu, United States. ⟨hal-04403681⟩
8 Consultations
12 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More