Improving performances of log mining for anomaly prediction through NLP-based log parsing - Archive ouverte HAL
Communication Dans Un Congrès Année : 2018

Improving performances of log mining for anomaly prediction through NLP-based log parsing

Résumé

Failure prediction of industrial systems is a promising application domain for data mining approaches and should naturally rely on log messages which are a prime source of data as they are generated by many systems. However, before extracting relevant information of such log messages, another critical step is to parse the logs, that is to say to transform a raw unstructured text from the log messages into a suitable input for data mining. These two problems (log parsing then log mining) are often studied separately while they are directly related in the context of failure prediction ; moreover, few performance benchmarks are publicly available. In this paper, we focus on the impact of log parsing techniques via natural language processing on the performances of log mining on two datasets. The first one is a log of an industrial aeronautical system comprising over 4, 500, 000 messages collected over one year of operation ; the second one is a public benchmark set from an HDFS cluster. On the latter, we show that it is possible to raise the F-score from 96% to 99.2% while using simpler and more robust log parsing techniques that require less parameter tuning provided that they are correctly combined with log mining techniques
Fichier principal
Vignette du fichier
2018_hal-01919820.pdf (161.76 Ko) Télécharger le fichier
Origine Fichiers éditeurs autorisés sur une archive ouverte

Dates et versions

hal-01919820 , version 1 (06-06-2023)

Identifiants

Citer

Nicolas Aussel, Yohan Petetin, Sophie Chabridon. Improving performances of log mining for anomaly prediction through NLP-based log parsing. MASCOTS 2018: 26th International Symposium on the Modeling, Analysis, and Simulation of Computer and Telecommunication Systems, Sep 2018, Milwaukee, United States. pp.237 - 243, ⟨10.1109/MASCOTS.2018.00031⟩. ⟨hal-01919820⟩
231 Consultations
269 Téléchargements

Altmetric

Partager

More