Discovering Editing Rules for Data Cleaning - Archive ouverte HAL
Communication Dans Un Congrès Année : 2012

Discovering Editing Rules for Data Cleaning

Thierno Diallo
  • Fonction : Auteur
Jean-Marc Petit
Sylvie Servigne

Résumé

Dirty data continues to be an important issue for companies. The database community pays a particular attention to this subject. A variety of integrity constraints like Conditional Functional Dependencies (CFD) have been studied for data cleaning. Data repair methods based on these constraints are strong to detect inconsistencies but are limited on how to correct data, worse they can even introduce new errors. Based on Master Data Management principles, a new class of data quality rules known as Editing Rules (eR) tells how to fix errors, pointing which attributes are wrong and what values they should take. However, finding data quality rules is an expensive process that involves intensive manual efforts. In this paper, we develop pattern mining techniques for discovering eRs from existing source relations (possibly dirty) with respect to master relations (supposed to be clean and accurate). In this setting, we propose a new semantics of eRs taking advantage of both source and master data. The problem turns out to be strongly related to the discovery of both CFD and one-to-one correspondences between sources and target attributes. We have proposed efficient techniques to address the discovery problem of eRs and heuristics to clean data. We have implemented and evaluated our techniques on real-life databases. Experiments show both the feasibility, the scalability and the robustness of our proposal.
Fichier principal
Vignette du fichier
QDB-VLDB_Editing_Rules-Discovery_ and_ Application_to_Data_Cleaning.pdf (601.17 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-01353088 , version 1 (04-12-2020)

Identifiants

  • HAL Id : hal-01353088 , version 1

Citer

Thierno Diallo, Jean-Marc Petit, Sylvie Servigne. Discovering Editing Rules for Data Cleaning. 10th International Workshop on Quality in Databases In conjunction with VLDB (Very Large Databases) 2012, Aug 2012, Istanbul, Turkey. pp.1-8. ⟨hal-01353088⟩
207 Consultations
106 Téléchargements

Partager

More