A Multilingual Dataset for Evaluating Parallel Sentence Extraction from Comparable Corpora - Archive ouverte HAL
Communication Dans Un Congrès Année : 2018

A Multilingual Dataset for Evaluating Parallel Sentence Extraction from Comparable Corpora

Serge Sharoff
  • Fonction : Auteur
  • PersonId : 1037571
Reinhard Rapp
  • Fonction : Auteur
  • PersonId : 1037572

Résumé

Comparable corpora can be seen as a reservoir for parallel sentences and phrases to overcome limitations in variety and quantity encountered in existing parallel corpora. This has motivated the design of methods to extract parallel sentences from comparable corporad. Despite this interest and work, no shared dataset has been made available for this task until the 2017 BUCC Shared Task. We present the challenges faced to build such a dataset and the solutions adopted to design and create the 2017 BUCC Shared Task dataset, emphasizing issues we had to cope with to include Chinese as one of the languages. The resulting corpus contains a total of about 3.5 million distinct sentences in English, French, German, Russian, and Chinese, mostly from Wikipedia. We illustrate the use of this dataset in the shared task and summarize the main results obtained by its participants. We finally outline remaining issues.
Fichier non déposé

Dates et versions

hal-01898362 , version 1 (18-10-2018)

Identifiants

  • HAL Id : hal-01898362 , version 1

Citer

Pierre Zweigenbaum, Serge Sharoff, Reinhard Rapp. A Multilingual Dataset for Evaluating Parallel Sentence Extraction from Comparable Corpora. International Conference on Language Resources and Evaluation, May 2018, Miyazaki, Japan. ⟨hal-01898362⟩
71 Consultations
0 Téléchargements

Partager

More