Collecting Tweets to Investigate Regional Variation in Canadian English - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2020

Collecting Tweets to Investigate Regional Variation in Canadian English

Filip Miletic
Ludovic Tanguy

Résumé

We present a 78.8-million-tweet, 1.3-billion-word corpus aimed at studying regional variation in Canadian English with a specific focus on the dialect regions of Toronto, Montreal, and Vancouver. Our data collection and filtering pipeline reflects complex design criteria, which aim to allow for both data-intensive modeling methods and user-level variationist sociolinguistic analysis. It specifically consists in identifying Twitter users from the three cities, crawling their entire timelines, filtering the collected data in terms of user location and tweet language, and automatically excluding near-duplicate content. The resulting corpus mirrors national and regional specificities of Canadian English, it provides sufficient aggregate and user-level data, and it maintains a reasonably balanced distribution of content across regions and users. The utility of this dataset is illustrated by two example applications: the detection of regional lexical and topical variation, and the identification of contact-induced semantic shifts using vector space models. In accordance with Twitter's developer policy, the corpus will be publicly released in the form of tweet IDs.
Fichier principal
Vignette du fichier
2020.lrec-1.767.pdf (557.78 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-02614672 , version 1 (21-05-2020)

Identifiants

  • HAL Id : hal-02614672 , version 1

Citer

Filip Miletic, Anne Przewozny-Desriaux, Ludovic Tanguy. Collecting Tweets to Investigate Regional Variation in Canadian English. 12th International Conference on Language Resources and Evaluation (LREC 2020), May 2020, Marseille, France. pp.6255-6264. ⟨hal-02614672⟩
116 Consultations
127 Téléchargements

Partager

Gmail Facebook X LinkedIn More