Creating Clustered Comparable Corpora from Wikipedia with Different Fuzziness Levels and Language Representativity - Archive ouverte HAL
Communication Dans Un Congrès Année : 2024

Creating Clustered Comparable Corpora from Wikipedia with Different Fuzziness Levels and Language Representativity

Résumé

This paper is dedicated to the extraction of clustered comparable corpora from Wikipedia, that is comparable corpora with labelled information corresponding to the topics associated to each document. Despite the importance of such corpora for evaluating text clustering and classification methods in the context of comparable corpora, there is a notable absence of automatic algorithms capable of creating them with adjustable fuzziness levels and language representativity. The methodology we propose here offers control over the cluster distribution across languages, enables fine-tuning of fuzziness levels, and facilitates customization to accommodate specific subject areas. Moreover, we have developed a dedicated tool specifically designed for our purpose and present 18 bilingual clustered comparable corpora spanning English, French, German, Russian, and Swedish languages. The analysis of these corpora demonstrates the effectiveness and flexibility of the approach in constructing corpora with varying levels of fuzziness and language representativity. Our results, tool and corpora, pave the way to construct various gold standard collections for future research in clustering and classification in comparable corpora.

Fichier principal
Vignette du fichier
2024.bucc-1.9.pdf (520.5 Ko) Télécharger le fichier
Origine Fichiers éditeurs autorisés sur une archive ouverte

Dates et versions

hal-04788316 , version 1 (18-11-2024)

Licence

Identifiants

  • HAL Id : hal-04788316 , version 1

Citer

Anna Laskina, Eric Gaussier, Gaelle Calvary. Creating Clustered Comparable Corpora from Wikipedia with Different Fuzziness Levels and Language Representativity. The 17th Workshop on Building and Using Comparable Corpora (BUCC)@ LREC-COLING 2024, May 2024, Torino, Italy. pp.85-93. ⟨hal-04788316⟩
11 Consultations
3 Téléchargements

Partager

More