CESAR: A new metric to measure the level of code-switching in corpora -Application to Maghrebian dialects - Archive ouverte HAL
Communication Dans Un Congrès Année : 2021

CESAR: A new metric to measure the level of code-switching in corpora -Application to Maghrebian dialects

Résumé

In this paper, we are interested in a sociolinguistic phenomenon that occurs in daily conversations of Maghrebi people, commonly known as code-switching or also code-mixing. This problem consists of alternating languages during communication or writing. In this work, we measure the importance of this phenomenon in the Maghrebi languages. To this end, we harvested from YouTube, comments written in Algerian, Moroccan and Tunisian dialects. Each of which contains at least 17 million words. Although there are several metrics in the literature to measure the code-switching, to the best of our knowledge, there isn't yet a measure that takes into account the degree of mixture according to a reference language. In contrast to the existing measures, we propose a new metric named CESAR (CodE-Switching According to a Reference language) that estimates the degree of the language mixtures, in accordance with a reference language. Experiments are carried out on the three collected corpora by considering the local dialects as reference languages. Experimental results show that CESAR is well adapted to this purpose and allows to compare the three Maghrebi dialects according to their level of code-switching.. . .
Fichier principal
Vignette du fichier
IntelliSys_2021_Papier_Cesar.pdf (279.02 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03139685 , version 1 (12-02-2021)

Identifiants

  • HAL Id : hal-03139685 , version 1

Citer

Karima Abidi, Kamel Smaïli. CESAR: A new metric to measure the level of code-switching in corpora -Application to Maghrebian dialects. IntelliSys2021, Sep 2021, Amsterdam, Netherlands. ⟨hal-03139685⟩
123 Consultations
432 Téléchargements

Partager

More