Cross-lingual and cross-domain evaluation of Machine Reading Comprehension with Squad and CALOR-Quest corpora - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020) Année : 2020

Cross-lingual and cross-domain evaluation of Machine Reading Comprehension with Squad and CALOR-Quest corpora

Résumé

Machine Reading received recently a lot of attention thanks to both the availability of very large corpora such as SQuAD or MS MARCO containing triplets (document, question, answer), and the introduction of Transformer Language Models such as BERT which obtains excellent results, even matching human performance according to the SQuAD leaderboard. One of the key features of Transformer Models is their ability to be jointly trained across multiple languages, using a shared subword vocabulary, leading to the construction of cross-lingual lexical representations. This feature has been used recently to perform zero-shot cross-lingual experiments where a multilingual BERT model fine-tuned on a machine reading comprehension task exclusively for English was directly applied to Chinese and French documents with interesting performance. In this paper we study the cross-language and cross-domain capabilities of BERT on a Machine Reading Comprehension task on two corpora: SQuAD and a new French Machine Reading dataset, called CALOR-QUEST. The semantic annotation available on CALOR-QUEST allows us to give a detailed analysis on the kind of questions that are properly handled through the cross-language process.
Fichier principal
Vignette du fichier
2020.lrec-1.674.pdf (316.81 Ko) Télécharger le fichier
Origine : Fichiers éditeurs autorisés sur une archive ouverte
Loading...

Dates et versions

hal-02973245 , version 1 (21-10-2020)

Identifiants

  • HAL Id : hal-02973245 , version 1

Citer

Delphine Charlet, Géraldine Damnati, Frédéric Bechet, Gabriel Marzinotto, Johannes Heinecke. Cross-lingual and cross-domain evaluation of Machine Reading Comprehension with Squad and CALOR-Quest corpora. LREC 2020, May 2020, MARSEILLE, France. pp.5491-5497. ⟨hal-02973245⟩
74 Consultations
44 Téléchargements

Partager

Gmail Facebook X LinkedIn More