Adapting Transformers for Multi-Label Text Classification - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

Adapting Transformers for Multi-Label Text Classification

Résumé

Pre-trained language models have proven to be effective in multi-class text classification. Our goal is to study and improve this approach for multi-label text classification, a task that has been surprisingly little explored in the last few years despite its many real world applications. In this paper, our originality is to propose architectures for the classification layers that are used on top of transformers to improve their performance for multi-label classification. Our contribution involves the evaluation of thresholding methods on several transformers, either by computing an individual threshold for each label (IT) or a global one (GCT). We also propose two approaches for multi-label text classification. The first consists in adding a parameter for learning the number of labels present for a given example (NHA). The second approach consists in adding a layer to the classification layers in order to learn the features for selecting the relevant labels while avoiding the use of thresholds (TL). We evaluate these approaches on two English corpora of newspaper articles and scientific papers and then on a new multi-label dataset of French scientific article abstracts publicly available. The evaluations show that the performance of our proposals exceeds that of state-of-the-art multi-label text classification methods for the evaluated datasets, and are transposable to any multi-label classification problem.
Fichier principal
Vignette du fichier
Adapting Transformers for Multi-Label Text Classification.pdf (1.18 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03727927 , version 1 (19-07-2022)

Identifiants

  • HAL Id : hal-03727927 , version 1

Citer

Haytame Fallah, Patrice Bellot, Emmanuel Bruno, Elisabeth Murisasco. Adapting Transformers for Multi-Label Text Classification. CIRCLE (Joint Conference of the Information Retrieval Communities in Europe) 2022, Jul 2022, Samatan, France. ⟨hal-03727927⟩
230 Consultations
588 Téléchargements

Partager

Gmail Facebook X LinkedIn More