Pré-Publication, Document De Travail (Preprint/Prepublication) Année : 2024

Splash: a Semi Structured Parser for Logs Assisted by Human Feedback

Sichao Yang
  • Fonction : Auteur
  • PersonId : 1438229

Résumé

During the runtime of a program, system statuses are normally documented in logs. Log parsing is a crucial initial step for subsequent analysis. While existing parsers have shown notable enhancements in accuracy when applied to public datasets, their performance is hindered notably when deployed in real-world settings due to two primary challenges. Firstly, the accuracy of these parsers is heavily reliant on hyperparameters, the optimal values of which vary significantly across different datasets. Secondly, the continuous software evolution along with its logging statements results in a rapid degradation of parsing accuracy after parser development. Although various mitigation strategies have been proposed, their efficacy remains limited. We propose a human-inthe-loop framework, named Splash, which integrates human expertise with data-driven methodologies. It operates on top of set of carefully designed parsing models and aims to minimize human effort by selfidentifying potential parsing errors and actively querying for human's assistance. Additionally, it employs proxy measures to monitor performance degradation and can request human assistance when necessary. Experimental evaluations conducted on 16 benchmark datasets demonstrate that our model outperforms all baselines in terms of accuracy. Furthermore, its effectiveness in mitigating model deterioration is exemplified through experiments conducted on a newly labeled dataset simulating a dynamic data environment.

Fichier principal
Vignette du fichier
paper.pdf (976.7 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Licence

Dates et versions

hal-04772637 , version 1 (08-11-2024)

Licence

Identifiants

  • HAL Id : hal-04772637 , version 1

Citer

Sichao Yang. Splash: a Semi Structured Parser for Logs Assisted by Human Feedback. 2024. ⟨hal-04772637⟩
35 Consultations
140 Téléchargements

Partager

  • More