Model-based Clustering with Missing Not At Random Data - Archive ouverte HAL
Pré-Publication, Document De Travail Année : 2021

Model-based Clustering with Missing Not At Random Data

Résumé

In recent decades, technological advances have made it possible to collect large data sets. In this context, the model-based clustering is a very popular, flexible and interpretable methodology for data exploration in a well-defined statistical framework. One of the ironies of the increase of large datasets is that missing values are more frequent. However, traditional ways (as discarding observations with missing values or imputation methods) are not designed for the clustering purpose. In addition, they rarely apply to the general case, though frequent in practice, of Missing Not At Random (MNAR) values, i.e. when the missingness depends on the unobserved data values and possibly on the observed data values. The goal of this paper is to propose a novel approach by embedding MNAR data directly within model-based clustering algorithms. We introduce a selection model for the joint distribution of data and missing-data indicator. It corresponds to a mixture model for the data distribution and a general MNAR model for the missing-data mechanism, which may depend on the underlying classes (unknown) and/or the values of the missing variables themselves. A large set of meaningful MNAR sub-models is derived and the identifiability of the parameters is studied for each of the sub-models, which is usually a key issue for any MNAR proposals. The EM and Stochastic EM algorithms are considered for estimation. Finally, we perform empirical evaluations for the proposed submodels on synthetic data and we illustrate the relevance of our method on a medical register, the TraumaBase ® dataset.
Fichier principal
Vignette du fichier
MNARclustering.pdf (774.25 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03494674 , version 1 (19-12-2021)
hal-03494674 , version 2 (13-05-2022)
hal-03494674 , version 3 (13-02-2023)
hal-03494674 , version 4 (21-12-2023)

Identifiants

Citer

Aude Sportisse, Christophe Biernacki, Claire Boyer, Julie Josse, Matthieu Marbac Lourdelle, et al.. Model-based Clustering with Missing Not At Random Data. 2021. ⟨hal-03494674v1⟩
743 Consultations
551 Téléchargements

Altmetric

Partager

More