Model-based Clustering with Missing Not At Random Data
Résumé
Model-based unsupervised learning, as any learning task, stalls as soon as
missing data occurs. This is even more true when the missing data are infor-
mative, or said missing not at random (MNAR). In this paper, we propose
model-based clustering algorithms designed to handle very general types
of missing data, including MNAR data. To do so, we introduce a mixture
model for different types of data (continuous, count, categorical and mixed)
to jointly model the data distribution and the MNAR mechanism, remaining
vigilant to the degrees of freedom of each. Eight different MNAR models
which depend on the class membership and/or on the values of the missing
variables themselves are proposed. For a particular type of MNAR mod-
els, for which the missingness depends on the class membership, we show
that the statistical inference can be carried out on the data matrix concate-
nated with the missing mask considering a MAR mechanism instead; this
specifically underlines the versatility of the studied MNAR models. Then,
we establish sufficient conditions for identifiability of parameters of both the
data distribution and the mechanism. Regardless of the type of data and the
mechanism, we propose to perform clustering using EM or stochastic EM
algorithms specially developed for the purpose. Finally, we assess the nu-
merical performances of the proposed methods on synthetic data and on the
real medical registry TraumaBase® as well.
Fichier principal
MNARclustering.pdf (878.27 Ko)
Télécharger le fichier
references.bib (88.99 Ko)
Télécharger le fichier
| Origine | Fichiers produits par l'(les) auteur(s) |
|---|