Splash: a Semi Structured Parser for Logs Assisted by Human Feedback
Résumé
During the runtime of a program, system statuses are normally documented in logs. Log parsing is a crucial initial step for subsequent analysis. While existing parsers have shown notable enhancements in accuracy when applied to public datasets, their performance is hindered notably when deployed in real-world settings due to two primary challenges. Firstly, the accuracy of these parsers is heavily reliant on hyperparameters, the optimal values of which vary significantly across different datasets. Secondly, the continuous software evolution along with its logging statements results in a rapid degradation of parsing accuracy after parser development. Although various mitigation strategies have been proposed, their efficacy remains limited. We propose a human-inthe-loop framework, named Splash, which integrates human expertise with data-driven methodologies. It operates on top of set of carefully designed parsing models and aims to minimize human effort by selfidentifying potential parsing errors and actively querying for human's assistance. Additionally, it employs proxy measures to monitor performance degradation and can request human assistance when necessary. Experimental evaluations conducted on 16 benchmark datasets demonstrate that our model outperforms all baselines in terms of accuracy. Furthermore, its effectiveness in mitigating model deterioration is exemplified through experiments conducted on a newly labeled dataset simulating a dynamic data environment.
| Origine | Fichiers produits par l'(les) auteur(s) |
|---|---|
| Licence |