ML-Based Anomaly Detection for Infrastructure Observability: An Intelligent AIOps Platform Using Ensemble Learning and LLM-Powered Root Cause Analysis

Abstract

ML-Based Anomaly Detection for Infrastructure Observability: An Intelligent AIOps Platform Using Ensemble Learning and LLM-Powered Root Cause Analysis Abstract This research proposes an intelligent AIOps platform designed to improve infrastructure observability by combining machine learning-based anomaly detection, multi-signal event correlation, and large language model-assisted root cause analysis. Traditional monitoring approaches often rely heavily on static thresholds, which can generate excessive alerts and require significant manual investigation. The proposed platform addresses these limitations by analyzing infrastructure metrics, logs, traces, and security-related events as complementary sources of operational context. The detection layer uses an ensemble of three complementary machine learning approaches: Prophet for time-series forecasting, LSTM for learning temporal patterns, and Isolation Forest for detecting statistical outliers. Their outputs are combined through an ensemble strategy to improve anomaly detection across different types of infrastructure behavior. Detected anomalies are then processed by a correlation engine that combines related metrics, logs, traces, and security events into higher-level incidents. The resulting incident context is passed to a locally deployed Qwen3-based large language model, which assists with root cause analysis, incident explanation, and investigation recommendations. The platform is built around an open and self-hosted observability stack consisting of Grafana Alloy, Prometheus, Loki, Tempo, and Grafana, with Kubernetes runtime security and audit signals incorporated as additional sources of operational context. The research will evaluate the proposed approach against conventional threshold-based monitoring and individual machine learning models. Evaluation will focus on detection precision, recall, false-positive rate, detection latency, correlation accuracy, and root-cause analysis performance. The expected outcome is a reproducible, cloud-native AIOps platform capable of transforming heterogeneous observability signals into actionable incident intelligence while maintaining control over infrastructure, data, and model deployment. Research Objectives Develop a multi-source infrastructure observability pipeline. Design an ensemble anomaly detection approach combining Prophet, LSTM, and Isolation Forest. Develop a correlation mechanism for connecting related observability and security events. Investigate the effectiveness of a locally deployed Qwen3-based LLM for root cause analysis. Compare the proposed approach with threshold-based and single-model detection methods. Evaluate the system using reproducible infrastructure failure and anomaly scenarios. Measure detection quality, false positives, detection latency, and RCA performance. Expected Research Contributions A cloud-native architecture integrating observability, security telemetry, ML-based anomaly detection, event correlation, and LLM-assisted RCA. An ensemble strategy for combining heterogeneous anomaly detection models. A multi-signal incident correlation methodology. A structured context extraction pipeline for LLM-based root cause analysis. An experimental evaluation framework for intelligent infrastructure observability. A reproducible self-hosted implementation suitable for research and experimentation.