skills.rest
curiositech
curiositechOFFICIAL
Explore all skills by this author →
WebsiteGitHub

observability-apm-expert

Guide distributed system observability with tracing, metrics, and structured logs.

2|Updated Feb 11, 2026
Search Tags:#metrics#observability#tracing#grafana#opentelemetry#datadog#apm
One-click install
npx skills add https://github.com/curiositech/port-daddy --skill observability-apm-expert
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: observability-apm-expert
Source: https://github.com/curiositech/port-daddy/tree/main/skills/observability-apm-expert
Command: npx skills add https://github.com/curiositech/port-daddy --skill observability-apm-expert

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Distributed systems often lack coherent telemetry, leaving teams blind to cross-service performance issues, missed error traces, and noisy alerts; this Skill provides structured guidance to design tracing, metrics, and logging so incidents are detectable and actionable.

Core Features & Use Cases

  • Sampling Strategy & Trace Retention: Tailored guidance to choose 100% error retention, tail-based sampling for high-cardinality services, and policies for slow-trace capture.
  • Backend Selection & Integration: Recommendations for self-hosted Grafana stack (Tempo, Mimir, Loki) or SaaS vendors (Datadog, Honeycomb) plus OTLP collector configuration and fallbacks.
  • Alerting and SLOs: SLO definition, error-budget calculation, burn-rate thresholds, and runbook linkage to reduce alert fatigue and speed response.
  • Incident Investigation Playbooks: Stepwise trace-first triage, correlation with infrastructure metrics, and remediation actions for common root causes like DB pool exhaustion.
  • Instrumentation Guidance: Practical advice for OpenTelemetry SDK usage, context propagation, log->trace correlation, and business-metric instrumentation with bounded cardinality.

Quick Start

Describe your service topology and ask for a recommended OpenTelemetry sampling strategy, backend choice, and the SLO/alert configuration to apply.

Frequently Asked Questions about observability-apm-expert

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I configure OpenTelemetry tail-based sampling for high-cardinality microservices?▼

SLO burn-rate alerting calculates error budget consumption against defined thresholds, triggering alerts only when reliability targets are genuinely threatened. This approach reduces alert fatigue by linking multi-window burn rates to runbooks for faster incident triage.

How do I correlate traces, logs, and metrics for distributed systems incident triage?▼

Trace-log-metric correlation requires OpenTelemetry context propagation across microservices to inject trace IDs into structured logs. This enables stepwise trace-first triage, linking infrastructure metrics and business metrics to pinpoint root causes like DB pool exhaustion.

What is the best observability backend for hybrid cloud deployments: self-hosted Grafana or Datadog?▼

Backend selection depends on your operational model: self-hosted Grafana stacks using Tempo, Mimir, and Loki offer control, while SaaS vendors like Datadog or Honeycomb reduce maintenance. Both integrate via OTLP collector configuration for telemetry ingestion.

How do I instrument OpenTelemetry SDK for bounded cardinality business metrics?▼

OpenTelemetry SDK instrumentation requires defining business metrics with bounded cardinality by limiting label combinations. This prevents unbounded metric explosion while maintaining trace-log correlation and context propagation across service mesh boundaries.

Does this observability guidance work with service mesh and microservice architectures?▼

Yes, this observability and APM guidance applies directly to microservices, service mesh, and hybrid cloud deployments. It supports OTLP exports, trace-log-metric correlation, and quality gates for distributed systems regardless of architectural complexity.

Related Skills

Explore More Software Engineering Skills→
wildbitcawildbitca
1

common-observability

Standardize logs, traces, and metrics across polyglot microservices with OpenTelemetry.

Official
Advanced
1k-off1k-off
1

alerting-irm

Provision Grafana alert rules, contact points, and notification policies via YAML.

Community
Intermediate
allthingslinuxallthingslinux
1

grafana

Design, configure, and deploy Grafana dashboards with LGTM stack observability.

Official
Advanced
patrykkopycinskipatrykkopycinski
6

migrate-from-datadog

Migrate Datadog APM instrumentation, traces, metrics, and logs to the Elastic Stack.

Community
Intermediate