Combining Machine Learning and Semantics for Anomaly Detection - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2018

Combining Machine Learning and Semantics for Anomaly Detection

Résumé

The emergence of the Internet of Things and stream processing forces large scale organizations to consider anomaly detection as a key component of their business. Using machine learning to solve such complex use cases is generally a cumbersome, costly, time-consuming and error-prone process. It involves many tasks from data cleansing, to dimension reduction, algorithm selection and fine tuning. It also requires the involvement of various experts such as statisticians, programmers and testers. With RAMSSES, we remove the burden of this pipeline and demonstrate that these tasks can be automated. Our system leverages on a Lambda architecture based on Apache Spark to analyze historical data, perform cleansing and deal with the curse of dimensionality. Then, it identifies the most interesting attributes and uses a continuous semantic query generator executed over streams. The sampled data are processed by self-selected machine learning methods to detect anomalies, an iterative process using end user annotations improves significantly the accuracy of the system. After a description of RAMSSES’s main components, the performance and relevancy of the system are demonstrated via a thorough evaluation over real-world and synthetic datasets.
Fichier non déposé

Dates et versions

hal-04491501 , version 1 (06-03-2024)

Identifiants

Citer

Badre Belabbess, Musab Bairat, Jeremy Lhez, Olivier Curé. Combining Machine Learning and Semantics for Anomaly Detection. EKAW 2018, Nov 2018, Nancy, France. pp.503-518, ⟨10.1007/978-3-030-03667-6_32⟩. ⟨hal-04491501⟩
12 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More