skills.rest
Lynricsy
LynricsyCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

observability

Instruments production systems with OpenTelemetry traces, metrics, logs, SLOs, and alerts.

22|Updated Sep 10, 2026
Search Tags:#observability#tracing#prometheus#alerting#opentelemetry#slo#cardinality
One-click install
npx skills add https://github.com/Lynricsy/HyperSkills --skill observability-lynricsy
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: observability
Source: https://github.com/Lynricsy/HyperSkills/tree/main/skills/observability
Command: npx skills add https://github.com/Lynricsy/HyperSkills --skill observability-lynricsy

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Production telemetry is easy to emit and hard to get right: high-cardinality labels silently corrupt rate() queries, misconfigured Collectors bind to loopback and receive nothing, head sampling discards the errors you need, and alerts fire on causes instead of symptoms. This Skill encodes the verified rules and workflows for instrumenting services, configuring OpenTelemetry Collector pipelines, controlling metric cardinality, and designing SLO-based alerting that pages only when users are hurt. ## Core Features & Use Cases - Instrumentation guidance: Span naming, kind and status rules, exception recording as log records, structured logging with trace correlation, and RED/USE metrics with bounded label sets. - Collector pipeline review: Processor ordering, memory_limiter semantics, agent versus gateway topology, tail sampling with load balancing, and validation recipes using otelcol validate and telemetrygen against otelcol-contrib 0.160.0. - Cardinality and alerting: TSDB status diagnosis, the rule against dropping distinguishing labels, histogram amplification, plus SLO anatomy, error budgets, and multi-window multi-burn-rate alert design. - Use Case: A service's Prometheus store is OOMing after a deploy added a user_id label. Use this Skill to diagnose the offending metric via the TSDB status endpoint, apply the safe emergency drop by name, and fix the source with an exemplar instead of a label. ## Quick Start Use the observability skill to review my OpenTelemetry Collector config and instrumentation code for correctness and cardinality risks.

Frequently Asked Questions about observability

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I instrument a service with OpenTelemetry traces and metrics?▼

Enable automatic instrumentation first, then add manual spans only for meaningful units of work with low-cardinality names and correct span kinds. Set resource identity once at the SDK, emit RED metrics per endpoint, and wrap headless entry points like cron jobs in explicit root spans.

How do I review an OpenTelemetry Collector configuration?▼

Establish the version first, then check that otlp receivers bind 0.0.0.0 explicitly, memory_limiter is first in every processor list, and nothing upserts service.name. Validate with otelcol validate, then run the Collector with telemetrygen and a debug exporter to confirm actual behavior.

Why does dropping a label at scrape time corrupt Prometheus metrics?▼

Removing a label that makes a series unique merges distinct series, so their counter resets interleave and rate() returns absurdly high values. No configuration error appears and no evidence remains in the data. Fix the source, drop the whole metric by __name__, or aggregate post-ingest instead.

Should I sample traces in the SDK or the Collector?▼

Leave the SDK sampler at AlwaysOn and sample in the Collector. Head sampling discards errors and latency outliers at the same rate as normal traffic, while tail sampling in a gateway tier can decide after the full trace is known. Materialize RED metrics before any sampling step.

How do I design alerts that page only for real user impact?▼

Alert on symptoms expressed as ratios and burn rates derived from an SLO, never on counts or causes like CPU. Use multi-window multi-burn-rate rules with a fast page and a slow ticket, give each paging alert a for duration, a runbook link, and one of two severities.

What does this observability skill not cover?▼

It excludes cloud monitoring products like CloudWatch and Azure Monitor, Elasticsearch index design, Kubernetes manifests for the Collector, and observability of AI agents via gen_ai conventions. It owns only the vendor-neutral layer: OTel instrumentation, Collector config, cardinality, alerting, and SLOs.

Related Skills

Explore More Software Engineering Skills→
Sir-chawakornSir-chawakorn
33

containerization

Create Dockerfiles, docker-compose configurations, and Kubernetes deployment manifests.

Community
Intermediate
m-de-graaffm-de-graaff

observability

Design structured logging, metrics, tracing, and alerting for production systems.

Community
Intermediate
heyAyushhheyAyushh
3

iterate-browser

Instrument browser pages with debug traces and reproduce frontend issues.

Community
Basic
dmarinsdmarins

observability-and-instrumentation

Implement structured logging, metrics, and distributed tracing with OpenTelemetry.

Community
Advanced