Topic models do not model topics: epistemological remarks and steps towards best practices - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Journal of Data Mining and Digital Humanities Année : 2021

Topic models do not model topics: epistemological remarks and steps towards best practices

Résumé

The social sciences and digital humanities have recently adopted the machine learning technique of topic modeling to address research questions in their fields. This is problematic in a number of ways, some of which have not received much attention in the debate yet. This paper adds epistemological concerns centering around the interface between topic modeling and linguistic concepts and the argumentative embedding of evidence obtained through topic modeling. It concludes that topic modeling in its present state of methodological integration does not meet the requirements of an independent research method. It operates from relevantly unrealistic assumptions, is non-deterministic, cannot effectively be validated against a reasonable number of competing models, does not lock into a well-defined linguistic interface, and does not scholarly model topics in the sense of themes or content. These features are intrinsic and make the interpretation of its results prone to apophenia (the human tendency to perceive random sets of elements as meaningful patterns) and confirmation bias (the human tendency to perceptually prefer patterns that are in alignment with pre-existing biases). While partial validation of the statistical model is possible, a conceptual validation would require an extended triangulation with other methods and human ratings, and clarification of whether statistical distinctivity of lexical co-occurrence correlates with conceputal topics in any reliable way.
Fichier principal
Vignette du fichier
shadrova_2021_topic_models_do_not_model_topics_final_formatted.pdf (308.68 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03261599 , version 1 (15-06-2021)
hal-03261599 , version 2 (13-08-2021)
hal-03261599 , version 3 (13-10-2021)

Licence

Paternité - Pas d'utilisation commerciale - Partage selon les Conditions Initiales

Identifiants

Citer

Anna Shadrova. Topic models do not model topics: epistemological remarks and steps towards best practices. Journal of Data Mining and Digital Humanities, 2021, 2021, ⟨10.46298/jdmdh.7595⟩. ⟨hal-03261599v3⟩
499 Consultations
1895 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More