LogRule: Efficient Structured Log Mining for Root Cause Analysis

Paolo Notaro, Soroush Haeri, Jorge Cardoso, Michael Gerndt

Research output: Contribution to journalArticlepeer-review

2 Scopus citations

Abstract

Accurate, timely Root Cause Analysis (RCA) is essential to successful IT operations as a primary step to incident remediation. RCA automation using data mining techniques in large heterogeneous systems is, however, a challenging task, because it requires correlating multimodal information across various data sources. An increasing number of services are migrating to structured logging to enable automated monitoring and debugging of complex large-scale systems. In this paper, we leverage structured logs and association rule mining (ARM) to automate RCA. We propose the LogRule algorithm, which automatically analyzes structured logs to generate a list of explanations for an event of interest. It achieves 0.921 F1-score for the diagnosis task, while computing results 37x faster compared to the state-of-the-art solution based on FP-growth, making it a time-efficient, accurate, and interpretable ARM-based RCA algorithm. Evaluation results show that LogRule enables RCA in complex multidimensional datasets, where the execution time of the current state-of-the-art algorithm is prohibitively large.

Original languageEnglish
Article number3282270
Pages (from-to)4231-4243
Number of pages13
JournalIEEE Transactions on Network and Service Management
Volume20
Issue number4
DOIs
StatePublished - 1 Dec 2023
Externally publishedYes

Keywords

  • AIOps
  • Root cause analysis
  • data mining
  • large-scale computing environment

Fingerprint

Dive into the research topics of 'LogRule: Efficient Structured Log Mining for Root Cause Analysis'. Together they form a unique fingerprint.

Cite this