monitoring-alerting

Monitoring and alerting automates metric collection, threshold detection, and notifications to catch issues before they impact users. Start by instrumenting your application with Prometheus client libraries to track counters, histograms, and gauges, then configure alert rules and notification channels like Slack or PagerDuty to trigger when thresholds breach.

Yes. Prometheus client integrates directly with FastAPI through middleware to automatically instrument HTTP request rate and latency. Grafana queries Prometheus metrics to build dashboards visualizing this data, enabling real-time performance monitoring without manual metric wiring in each endpoint.

SLIs (Service Level Indicators) measure actual system performance like error rate or latency. SLOs (Service Level Objectives) are targets you set—for example, 99.9% availability. SLAs (Service Level Agreements) are commitments to customers with penalties. This skill helps define SLIs with Prometheus queries and set SLOs to track your error budget.

Error budgets quantify how much failure you can tolerate within an SLO window. Define your SLO as a success rate target, calculate remaining budget in Prometheus, and create alerts that fire when budget exhaustion approaches, preventing reliability degradation before it violates your commitment.

Instrument request count, latency (histograms), error rates, and business metrics relevant to your SLO. For microservices, track per-endpoint metrics using Prometheus counters and gauges, expose them via FastAPI, and correlate with downstream service dependencies to diagnose failures end-to-end.

This skill provides patterns to generate dashboards programmatically from your SLO and SLI definitions, reducing manual setup. Dashboards query Prometheus metrics automatically, display error budgets, alert status, and latency trends, giving operators visibility without dashboard rebuilding each release.