alert-investigator

Correlate Prometheus metrics and Loki logs to determine alert root causes.

2|Updated Feb 5, 2026
One-click install
npx skills add https://github.com/damacus/agent-skills --skill alert-investigator
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: alert-investigator
Source: https://github.com/damacus/agent-skills/tree/main/skills/alert-investigator
Command: npx skills add https://github.com/damacus/agent-skills --skill alert-investigator

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Expert Site Reliability Engineer (SRE) specialized in deep-dive investigation of firing infrastructure alerts. Use this agent whenever Alertmanager or Grafana alerts are active and you need a comprehensive root-cause analysis. It should be used to investigate firing alerts by correlating Prometheus metrics and Loki logs, detecting error patterns, analyzing slow requests, and inspecting Kubernetes resource states to provide actionable remediation steps.

Core Features & Use Cases

  • Deep alert analysis: identify firing alerts, triage with annotations, and correlate metrics, logs, and state.
  • End-to-end investigation: analyze Prometheus metrics, Loki logs, traces, and Kubernetes state to determine root cause.
  • Remediation guidance: produce concrete steps to restore service health and prevent recurrence.

Quick Start

Identify an active firing alert and begin a deep-dive investigation using Grafana/Prometheus/Loki data.

Frequently Asked Questions about alert-investigator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I investigate firing Kubernetes alerts using Prometheus and Loki?

Investigating firing alerts involves querying Prometheus for metrics and Loki for logs to detect error patterns, analyze slow requests, inspect Kubernetes resource states, and produce actionable remediation steps to restore service health.

What is root-cause analysis for Grafana Alertmanager alerts?

Root-cause analysis for active alerts is a structured workflow that triages firing alerts using their annotations, correlates metrics, logs, traces, and Kubernetes state, and outputs a detailed investigation report with concrete remediation steps.

Can I use this to correlate Loki logs with Prometheus metrics during an incident?

Yes, you can correlate Loki logs with Prometheus metrics during an incident by querying both data sources to detect error patterns and analyze slow requests, inspecting Kubernetes state, and generating a comprehensive investigation report.

Does alert-investigator work with my existing Grafana alerting setup?

It works with Grafana alerting setups by utilizing Grafana, Prometheus, and Loki tooling such as list_alert_rules, query_prometheus, and query_loki_logs to identify active alerts and perform end-to-end deep-dive investigations.

What is the best way to triage active infrastructure alerts in Kubernetes?

The best way to triage active alerts is to apply a structured workflow that identifies firing alerts, analyzes their annotations, and correlates Prometheus metrics, Loki logs, traces, and Kubernetes state to determine root causes and provide remediation guidance.

How do I get actionable remediation steps for firing infrastructure alerts?

You get actionable remediation steps for firing alerts by performing an end-to-end investigation that analyzes Prometheus metrics, Loki logs, traces, and Kubernetes resource states to determine the root cause and restore service health.