observability-and-instrumentation

Instruments production code with structured logging, metrics, tracing, and alerting.

Updated Jul 22, 2026
One-click install
npx skills add https://github.com/Chau165/local_skill --skill observability-and-instrumentation-chau165
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: observability-and-instrumentation
Source: https://github.com/Chau165/local_skill/tree/main/codex/skills/observability-and-instrumentation
Command: npx skills add https://github.com/Chau165/local_skill --skill observability-and-instrumentation-chau165

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Production features often ship without telemetry, so when incidents occur engineers cannot tell what happened from the available data. This Skill guides you to instrument code alongside the feature so production behavior is visible and diagnosable from day one. ## Core Features & Use Cases - Structured Logging: Enforces JSON log events with stable event names, consistent log levels, and mandatory correlation/request IDs propagated across services. - Metrics and Tracing: Applies RED (Rate, Errors, Duration) and USE (Utilization, Saturation, Errors) methods with bounded label cardinality, plus OpenTelemetry-based distributed tracing with context propagation. - Symptom-Based Alerting: Designs actionable alerts tied to user-facing symptoms with runbook links, and verifies telemetry by inducing failures in staging. - Use Case: When adding a payment retry flow, use this Skill to define the on-call questions first, then add structured payment_failed log events, a latency histogram for the provider, trace spans across services, and an error-rate alert with a runbook. ## Quick Start Use the observability-and-instrumentation skill to add structured logging, RED metrics, tracing, and a symptom-based alert to my new checkout endpoint.

Frequently Asked Questions about observability-and-instrumentation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I add structured logging to a production service?

Structured logging means emitting JSON log events with stable event names and machine-readable fields instead of interpolated strings. Attach a correlation or request ID generated at the system boundary to every log line, and use consistent levels: error, warn, info, and debug.

What metrics should I instrument for HTTP endpoints and external dependencies?

Use the RED method for request-driven services: Rate, Errors, and Duration as a latency histogram on every endpoint and external dependency. For resources like queues and pools, use USE: Utilization, Saturation, and Errors. Track percentiles like p95 and p99, never averages.

Should I use OpenTelemetry or Prometheus for observability?

They serve different signals: OpenTelemetry is the vendor-neutral standard for distributed tracing and metrics APIs, while Prometheus prom-client is one common metrics backend. The RED/USE and cardinality rules apply identically regardless of which backend you choose.

Why is high label cardinality a problem in metrics?

Every unique label combination creates a separate time series, so unbounded values like user IDs, raw URLs, or error messages can overwhelm the metrics backend. Labels must come from small fixed sets such as route templates, status classes like 5xx, and provider names.

When should I not use this observability skill?

Do not use it to diagnose an active failure; that belongs to a debugging skill which this instrumentation makes faster next time. It also does not cover performance profiling of measured slowness or launch-day monitoring checklists, which belong to release workflows.