Reinforcement Learning for Data Cleaning and Data Preparation
Résumé
Data cleaning and data preparation have been long-standing challenges in data science to avoid incorrect results, biases, and misleading conclusions obtained from "dirty" data. For a given dataset and a given analytics task, a plethora of data preprocessing techniques and alternative data cleaning strategies are available, but they may lead to dramatically different outputs with unequal ML model quality performances. For adequate data preparation, the users generally do not know how to start or which methods to use. Most current work focus either on proposing new data cleaning algorithms -often specific to certain types of data glitches considered in isolation and generally with no "pipeline vision" of the whole data preprocessing sequence-or on developing automated machine learning approaches (AutoML) that can optimize the hyper-parameters of the given ML model but that often rely on by-default preprocessing methods. We argue that more efforts should be devoted to proposing a principled data preparation approach to help and learn from the users for selecting the optimal sequence of data curation tasks and obtain the best quality performance of the final result. In this abstract, we present Learn2Clean 1 , a method based on Q-Learning, a model-free reinforcement learning technique that selects, for a given dataset, a given ML model, and a quality performance metric, the optimal sequence of tasks for preprocessing the data such that the quality metric is maximized. Learn2Clean has been presented in The Web Conf 2019 [1] and we will discuss Learn2Clean enhancements for semi-automated data preparation guided by the user.
Domaines
| Origine | Fichiers produits par l'(les) auteur(s) |
|---|---|
| Licence |