investigate-alert

Investigate firing Prometheus alerts by gathering pod health, events, and Kibana logs.

2|Updated Feb 16, 2026

Search Tags:#debugging #kubernetes #incident response #prometheus #alerting #triage

Ready to run

npx skills add https://github.com/dmzoneill/skills --skill investigate-alert

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: investigate-alert
Source: https://github.com/dmzoneill/skills/tree/main/investigate-alert
Command: npx skills add https://github.com/dmzoneill/skills --skill investigate-alert

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill rapidly triages firing Prometheus alerts by gathering essential context like pod health, events, and logs, enabling quick assessment and escalation.

Core Features & Use Cases

Alert Triage: Automatically fetches firing alerts from Prometheus.
System Health Check: Assesses Kubernetes pod health, resource usage, and recent events.
Log Analysis: Searches Kibana for relevant errors and exceptions.
Pattern Matching: Compares issues against known patterns for faster diagnosis.
Escalation: Automatically escalates critical alerts to debug_prod if necessary.
Use Case: When a critical alert fires, this Skill provides a comprehensive overview of the system's status, helping to pinpoint the root cause without manual intervention.

Quick Start

Investigate the firing Prometheus alerts for the 'stage' environment and 'main' namespace.

Frequently Asked Questions about investigate-alert

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I automatically triage firing Prometheus alerts in Kubernetes?▼

You can automatically triage firing Prometheus alerts by gathering Kubernetes pod health, events, and Kibana logs. This approach assesses alert severity and auto-escalates critical issues to debug_prod without manual intervention.

What does an automated incident response workflow for Prometheus alerting look like?▼

An automated incident response workflow fetches firing Prometheus alerts, checks Kubernetes pod health, searches Kibana logs for errors, and matches issues against known patterns. It then evaluates alert severity to determine if escalation is required.

How do I investigate Kubernetes pod health and Kibana logs when an alert fires?▼

To investigate alert context, you gather Kubernetes pod health, resource usage, and recent events alongside Kibana logs. Comparing these details against known patterns enables faster diagnosis of the root cause.

Do I need kubeconfig and Kibana access to investigate Prometheus alerts?▼

Yes, you need an incident persona, kubeconfig rules, and access to Prometheus, Kibana, and Kubernetes tools. These dependencies are required to fetch alerts, query logs, and assess pod health accurately.

Can I auto-escalate critical Kubernetes alerts to debug_prod?▼

Yes, you can auto-escalate critical Prometheus alerts to debug_prod for serious problems. The Skill assesses alert severity during triage and automatically triggers this escalation when critical issues are detected.

What are the limitations of automated Kubernetes alert triage and pattern matching?▼

Automated alert triage relies on comparing issues against known patterns, which may miss novel anomalies. It requires pre-configured kubeconfig rules and access to Prometheus, Kibana, and Kubernetes tools to function properly.