Validation of wastewater data using artificial intelligence tools and the evaluation of their performance regarding annotator agreement - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Water Science and Technology Année : 2023

Validation of wastewater data using artificial intelligence tools and the evaluation of their performance regarding annotator agreement

Résumé

To prevent the pollution of water resources, the measurement and the limitation of wastewater discharges are required. Despite the progress in the field of data acquisition systems, sensors are subject to malfunctions that can bias the evaluation of the pollution flow. It is therefore essential to identify potential anomalies in the data before any use. The objective of this work is to deploy artificial intelligence tools to automate the data validation and to assess the added value of this approach in assisting the validation performed by an operator. To do so, we compare two state-of-the-art anomaly detection algorithms on turbidity data in a sewer network. On the one hand, we conclude that the One-class SVM model is not adapted to the nature of the studied data which is heterogeneous and noisy. The Matrix Profile model, on the other hand, provides promising results with a majority of anomalies detected and a relatively limited number of false positives. By comparing these results to the expert validation, it turns out that the use of the Matrix Profile model objectifies and accelerates the validation task while maintaining the same level of performance compared to the annotator agreement rate between two experts.
Fichier principal
Vignette du fichier
wst2023174.pdf (1.05 Mo) Télécharger le fichier
Origine : Fichiers éditeurs autorisés sur une archive ouverte
Licence : CC BY NC ND - Paternité - Pas d'utilisation commerciale - Pas de modification

Dates et versions

hal-04116074 , version 1 (28-06-2023)

Identifiants

Citer

Imane Zidaoui, Cédric Wemmert, Matthieu Dufresne, Claude Joannis, Sandra Isel, et al.. Validation of wastewater data using artificial intelligence tools and the evaluation of their performance regarding annotator agreement. Water Science and Technology, inPress, 87 (12), ⟨10.2166/wst.2023.174⟩. ⟨hal-04116074⟩
44 Consultations
40 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More