Towards a Robust Detection of Language Model-Generated Text: Is ChatGPT that easy to detect? - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Towards a Robust Detection of Language Model-Generated Text: Is ChatGPT that easy to detect?

Résumé

Recent advances in natural language processing (NLP) have led to the development of large language models (LLMs) such as ChatGPT. This paper proposes a methodology for developing and evaluating ChatGPT detectors for French text, with a focus on investigating their robustness on out-of-domain data and against common attack schemes. The proposed method involves translating an English dataset into French and training a classifier on the translated data. Results show that the detectors can effectively detect ChatGPT-generated text, with a degree of robustness against basic attack techniques in in-domain settings. However, vulnerabilities are evident in out-of-domain contexts, highlighting the challenge of detecting adversarial text. The study emphasizes caution when applying in-domain testing results to a wider variety of content. We provide our translated datasets and models as open-source resources.
Fichier principal
Vignette du fichier
461938.pdf (228.73 Ko) Télécharger le fichier
Origine : Fichiers éditeurs autorisés sur une archive ouverte

Dates et versions

hal-04130146 , version 1 (20-06-2023)

Identifiants

  • HAL Id : hal-04130146 , version 1

Citer

Wissam Antoun, Virginie Mouilleron, Benoît Sagot, Djamé Seddah. Towards a Robust Detection of Language Model-Generated Text: Is ChatGPT that easy to detect?. 18e Conférence en Recherche d'Information et Applications -- 16e Rencontres Jeunes Chercheurs en RI -- 30e Conférence sur le Traitement Automatique des Langues Naturelles -- 25e Rencontre des Étudiants Chercheurs en Informatique pour le Traitement Automatique des Langues, Jun 2023, Paris, France. pp.14-27. ⟨hal-04130146⟩
60 Consultations
29 Téléchargements

Partager

Gmail Facebook X LinkedIn More