EneBERT: A State-of-the-art Language Model Trained on a Corpus of Texts Generated from the Set of DSO Activities
Résumé
Pre-trained language model based on the Transformers architecture has significantly advanced natural language processing. This allowed for better results on several tasks, such as text classification or named entity recognition. Most existing models are general, and their use in a particular domain is sometimes tedious. Indeed, some domains have terms that are outside of commonly written languages. In these conditions, the implementation of a domain-specific model is necessary. In this paper, we have introduced a language model for the DSO activities of Enedis. This model is based on state-of-the-art language models but has been trained on domain-specific data. We also show that using the model for specific tasks, such as text classification, increases the performance obtained compared to a general language model.
Origine : Fichiers produits par l'(les) auteur(s)