prometheus-grafana-triage

Classify Prometheus, Grafana, and Alertmanager alerts to distinguish real platform issues from scrape failures.

Updated Mar 18, 2026

Search Tags:#devops #monitoring #observability #grafana #prometheus #triage #alertmanager

Ready to run

npx skills add https://github.com/Canepro/codex-skills --skill prometheus-grafana-triage

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: prometheus-grafana-triage
Source: https://github.com/Canepro/codex-skills/tree/main/skills/prometheus-grafana-triage
Command: npx skills add https://github.com/Canepro/codex-skills --skill prometheus-grafana-triage

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

Prometheus Grafana Triage helps operators quickly differentiate real platform incidents from scrape failures and bad rules, reducing MTTR and escalation noise.

Core Features & Use Cases

Classifies alerts, scrape health, and rule logic to determine the root cause.
Provides guidance for hub-and-spoke Prometheus setups and multi-cluster environments.
Useful for on-call incident response, post-incident reviews, and monitoring hygiene.

Quick Start

Run the triage workflow against your monitoring stack to classify the current incident.

Frequently Asked Questions about prometheus-grafana-triage

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I triage Prometheus alerts to distinguish real incidents from scrape failures?▼

To triage Prometheus alerts, you classify alerts and monitoring signals to differentiate real platform issues from scrape failures and bad rules. This process analyzes live data to reduce escalation noise and identify root causes.

Why does my Alertmanager keep firing noisy alerts for down targets?▼

Noisy Alertmanager alerts often stem from bad PromQL rules or Prometheus scrape failures. Analyzing alert classifications and scrape health helps distinguish genuine platform incidents from stale dashboards or incorrect rule logic.

Does this triage workflow require access to live Prometheus and Grafana endpoints?▼

Yes, this triage workflow requires access to live Prometheus, Alertmanager, and Grafana endpoints. It uses local helper scripts to analyze live monitoring data and classify the current active incident.

What is the best way to classify monitoring signals in a multi-cluster Prometheus setup?▼

To classify monitoring signals in multi-cluster environments, apply a triage workflow that evaluates alert logic and scrape health across hub-and-spoke Prometheus setups to determine root causes and improve monitoring hygiene.

How do I debug incorrect PromQL logic causing stale Grafana dashboards?▼

To debug stale Grafana dashboards, classify alert rules and monitoring signals to identify incorrect PromQL logic. Triage workflows evaluate rule logic against live data to pinpoint scrape failures or bad rules.

When should I not use an automated triage workflow for observability incidents?▼

Automated triage workflows are not suitable when you lack endpoint access to Prometheus, Alertmanager, and Grafana, or when you need to manually trace complex application-level failures unrelated to scrape failures or bad rules.