Interactive anomaly detection in mixed tabular data using Bayesian networks - Archive ouverte HAL
Communication Dans Un Congrès Année : 2020

Interactive anomaly detection in mixed tabular data using Bayesian networks

Résumé

The last decades improvements in processing abilities have quickly led to an increasing use of data analyses implying massive data-sets. To retrieve insightful information from any data driven approach, a pivotal aspect to ensure is good data quality. Manual correction of massive data-sets requires tremendous efforts, is prone to errors, and results being really costly. If knowledge in a specific field can often allow the development of efficient models for anomaly detection and data correction, this knowledge can sometimes be unavailable and a more generic approach should be found. This paper presents a novel approach to anomaly detection and correction in mixed tabular data using Bayesian Networks. We present an algorithm for detecting anomalies and offering correction hints based on Jensen scores computed within the Markov Blankets of considered variables. We also discuss the incremental corrections of detection model using user's feedback, as well as additional aspects related to discretization in mixed data and its effects on detection efficiency. Finally we also discuss how functional dependencies can be managed to detect errors while improving faithfulness and computation speed.
Fichier principal
Vignette du fichier
dufraisse20a.pdf (451.08 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03014622 , version 1 (17-03-2021)

Identifiants

  • HAL Id : hal-03014622 , version 1

Citer

Evan Dufraisse, Philippe Leray, Raphaël Nedellec, Tarek Benkhelif. Interactive anomaly detection in mixed tabular data using Bayesian networks. 10th International Conference on Probabilistic Graphical Models (PGM 2020), Sep 2020, Aalborg, Denmark. ⟨hal-03014622⟩
172 Consultations
243 Téléchargements

Partager

More