A Multilingual, Multi-Style and Multi-Granularity Dataset for Cross-Language Textual Similarity Detection - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2016

A Multilingual, Multi-Style and Multi-Granularity Dataset for Cross-Language Textual Similarity Detection

Résumé

In this paper we describe our effort to create a dataset for the evaluation of cross-language textual similarity detection. We present pre-existing corpora and their limits and we explain the various gathered resources to overcome these limits and build our enriched dataset. The proposed dataset is multilingual, includes cross-language alignment for different granularities (from chunk to document), is based on both parallel and comparable corpora and contains human and machine translated texts. Moreover, it includes texts written by multiple types of authors (from average to professionals). With the obtained dataset, we conduct a systematic and rigorous evaluation of several state-of-the-art cross-language textual similarity detection methods. The evaluation results are reviewed and discussed. Finally, dataset and scripts are made publicly available on GitHub: http://github.com/FerreroJeremy/Cross-Language-Dataset.
Fichier principal
Vignette du fichier
xample.pdf (216.61 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01303135 , version 1 (15-04-2016)

Identifiants

  • HAL Id : hal-01303135 , version 1

Citer

Jérémy Ferrero, Frédéric Agnès, Laurent Besacier, Didier Schwab. A Multilingual, Multi-Style and Multi-Granularity Dataset for Cross-Language Textual Similarity Detection. 10th edition of the Language Resources and Evaluation Conference, May 2016, Portorož, Slovenia. ⟨hal-01303135⟩
279 Consultations
484 Téléchargements

Partager

Gmail Facebook X LinkedIn More