Collecting Tweets to Investigate Regional Variation in Canadian English - Archive ouverte HAL Access content directly
Conference Papers Year : 2020

Collecting Tweets to Investigate Regional Variation in Canadian English

Abstract

We present a 78.8-million-tweet, 1.3-billion-word corpus aimed at studying regional variation in Canadian English with a specific focus on the dialect regions of Toronto, Montreal, and Vancouver. Our data collection and filtering pipeline reflects complex design criteria, which aim to allow for both data-intensive modeling methods and user-level variationist sociolinguistic analysis. It specifically consists in identifying Twitter users from the three cities, crawling their entire timelines, filtering the collected data in terms of user location and tweet language, and automatically excluding near-duplicate content. The resulting corpus mirrors national and regional specificities of Canadian English, it provides sufficient aggregate and user-level data, and it maintains a reasonably balanced distribution of content across regions and users. The utility of this dataset is illustrated by two example applications: the detection of regional lexical and topical variation, and the identification of contact-induced semantic shifts using vector space models. In accordance with Twitter's developer policy, the corpus will be publicly released in the form of tweet IDs.
Fichier principal
Vignette du fichier
2020.lrec-1.767.pdf (557.78 Ko) Télécharger le fichier
Origin Files produced by the author(s)
Loading...

Dates and versions

hal-02614672 , version 1 (21-05-2020)

Identifiers

  • HAL Id : hal-02614672 , version 1

Cite

Filip Miletic, Anne Przewozny-Desriaux, Ludovic Tanguy. Collecting Tweets to Investigate Regional Variation in Canadian English. 12th International Conference on Language Resources and Evaluation (LREC 2020), May 2020, Marseille, France. pp.6255-6264. ⟨hal-02614672⟩
131 View
143 Download

Share

Gmail Mastodon Facebook X LinkedIn More