Transductive Learning for Textual Few-Shot Classification in API-based Embedding Models - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Transductive Learning for Textual Few-Shot Classification in API-based Embedding Models

Résumé

Proprietary and closed APIs are becoming increasingly common to process natural language, and are impacting the practical applications of natural language processing, including fewshot classification. Few-shot classification involves training a model to perform a new classification task with a handful of labeled data. This paper presents three contributions. First, we introduce a scenario where the embedding of a pre-trained model is served through a gated API with compute-cost and data-privacy constraints. Second, we propose a transductive inference, a learning paradigm that has been overlooked by the NLP community. Transductive inference, unlike traditional inductive learning, leverages the statistics of unlabeled data. We also introduce a new parameter-free transductive regularizer based on the Fisher-Rao loss, which can be used on top of the gated API embeddings. This method fully utilizes unlabeled data, does not share any label with the third-party API provider and could serve as a baseline for future research. Third, we propose an improved experimental setting and compile a benchmark of eight datasets involving multiclass classification in four different languages, with up to 151 classes. We evaluate our methods using eight backbone models, along with an episodic evaluation over 1,000 episodes, which demonstrate the superiority of transductive inference over the standard inductive setting.
Fichier principal
Vignette du fichier
Colombo2023Transductive.pdf (1.49 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04327170 , version 1 (06-12-2023)

Identifiants

Citer

Pierre Colombo, Victor Pellegrain, Malik Boudiaf, Victor Storchan, Myriam Tami, et al.. Transductive Learning for Textual Few-Shot Classification in API-based Embedding Models. Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023, Singapore, Singapore. ⟨hal-04327170⟩
58 Consultations
4 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More