Powered Dirichlet Process - Controlling the "Rich-Get-Richer" Assumption in Bayesian Clustering - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Powered Dirichlet Process - Controlling the "Rich-Get-Richer" Assumption in Bayesian Clustering

Résumé

The Dirichlet process is one of the most widely used priors in Bayesian clustering. This process allows for a nonparametric estimation of the number of clusters when partitioning datasets. The "rich-getricher" property is a key feature of this process, and transcribes that the a priori probability for a cluster to get selected dependent linearly on its population. In this paper, we show that such hypothesis is not necessarily optimal. We derive the Powered Dirichlet Process as a generalization of the Dirichlet-Multinomial distribution as an answer to this problem. We then derive some of its fundamental properties (expected number of clusters, convergence). Unlike state-of-the-art efforts in this direction, this new formulation allows for direct control of the importance of the "rich-get-richer" prior. We confront our proposition to several simulated and real-world datasets, and confirm that our formulation allows for significantly better results in both cases.
Fichier principal
Vignette du fichier
Accepted___ECMLPKDD_23___PDP (1).pdf (4.06 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04171235 , version 1 (26-07-2023)

Identifiants

  • HAL Id : hal-04171235 , version 1

Citer

Gaël Poux-Médard, Julien Velcin, Sabine Loudcher. Powered Dirichlet Process - Controlling the "Rich-Get-Richer" Assumption in Bayesian Clustering. ECML-PKDD 2023, Sep 2023, Torino, Italy. ⟨hal-04171235⟩
20 Consultations
23 Téléchargements

Partager

Gmail Facebook X LinkedIn More