EneBERT: A State-of-the-art Language Model Trained on a Corpus of Texts Generated from the Set of DSO Activities - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

EneBERT: A State-of-the-art Language Model Trained on a Corpus of Texts Generated from the Set of DSO Activities

Romain Gemignani
  • Fonction : Auteur
  • PersonId : 1269532

Résumé

Pre-trained language model based on the Transformers architecture has significantly advanced natural language processing. This allowed for better results on several tasks, such as text classification or named entity recognition. Most existing models are general, and their use in a particular domain is sometimes tedious. Indeed, some domains have terms that are outside of commonly written languages. In these conditions, the implementation of a domain-specific model is necessary. In this paper, we have introduced a language model for the DSO activities of Enedis. This model is based on state-of-the-art language models but has been trained on domain-specific data. We also show that using the model for specific tasks, such as text classification, increases the performance obtained compared to a general language model.
Fichier principal
Vignette du fichier
CIRED Enebert.pdf (474.11 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04159669 , version 1 (27-07-2023)

Identifiants

  • HAL Id : hal-04159669 , version 1

Citer

Eunice Akani, Romain Gemignani, Rim Abrougui. EneBERT: A State-of-the-art Language Model Trained on a Corpus of Texts Generated from the Set of DSO Activities. International Conference & Exhibition on Electricity Distribution (CIRED 2023), Jun 2023, Rome, Italy. ⟨hal-04159669⟩
60 Consultations
55 Téléchargements

Partager

Gmail Facebook X LinkedIn More