Fast and fully-automated histograms for large-scale data sets - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Computational Statistics and Data Analysis Année : 2023

Fast and fully-automated histograms for large-scale data sets

Résumé

G-Enum histograms are a new fast and fully automated method for irregular histogram construction. By framing histogram construction as a density estimation problem and its automation as a model selection task, these histograms leverage the Minimum Description Length principle (MDL) to derive two different model selection criteria. Several proven theoretical results about these criteria give insights about their asymptotic behavior and are used to speed up their optimisation. These insights, combined to a greedy search heuristic, are used to construct histograms in linearithmic time rather than the polynomial time incurred by previous works. The capabilities of the proposed MDL density estimation method are illustrated with reference to other fully automated methods in the literature, both on synthetic and large real-world data sets.
Fichier principal
Vignette du fichier
MDL_histograms.pdf (3.44 Mo) Télécharger le fichier
MDL_histograms.spl (0 B) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03909919 , version 1 (22-12-2022)

Licence

Paternité

Identifiants

Citer

Valentina Zelaya Mendizábal, Marc Boullé, Fabrice Rossi. Fast and fully-automated histograms for large-scale data sets. Computational Statistics and Data Analysis, 2023, 180, pp.107668. ⟨10.1016/j.csda.2022.107668⟩. ⟨hal-03909919⟩
120 Consultations
99 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More