Definitions Matter: Guiding GPT for Multi-label Classification - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Definitions Matter: Guiding GPT for Multi-label Classification

Résumé

Large language models have recently risen in popularity due to their ability to perform many natural language tasks without requiring any fine-tuning. In this work, we focus on two novel ideas: (1) generating definitions from examples and using them for zero-shot classification, and (2) investigating how an LLM makes use of the definitions. We thoroughly analyze the performance of GPT-3 model for fine-grained multi-label conspiracy theory classification of tweets using zero-shot labeling. In doing so, we asses how to improve the labeling by providing minimal but meaningful context in the form of the definitions of the labels. We compare descriptive noun phrases, humancrafted definitions, introduce a new method to help the model generate definitions from examples, and propose a method to evaluate GPT-3's understanding of the definitions. We demonstrate that improving definitions of class labels has a direct consequence on the downstream classification results.
Fichier principal
Vignette du fichier
2023.findings-emnlp.267.pdf (247.84 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04346496 , version 1 (19-12-2023)

Identifiants

Citer

Youri Peskine, Damir Korenčić, Ivan Grubisic, Paolo Papotti, Raphael Troncy, et al.. Definitions Matter: Guiding GPT for Multi-label Classification. EMNLP 2023, Conference on Empirical Methods in Natural Language Processing, ACL, Dec 2023, Singapore, Singapore. ⟨10.18653/v1/2023.findings-emnlp.267⟩. ⟨hal-04346496⟩

Collections

EURECOM ANR
36 Consultations
18 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More