infra-observability-stack-review

Design and audit Kubernetes observability stacks using LGTM with OpenTelemetry Collector.

14|1|Updated May 5, 2026
One-click install
npx skills add https://github.com/ivanshamaev/de-agent-skills --skill infra-observability-stack-review
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: infra-observability-stack-review
Source: https://github.com/ivanshamaev/de-agent-skills/tree/main/group_skills/infra_dataops_group_skills/infra_observability_stack_review
Command: npx skills add https://github.com/ivanshamaev/de-agent-skills --skill infra-observability-stack-review

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Обеспечивает аудит и проектирование observability-стека для data platform так, чтобы метрики, логи и трассировки работали согласованно, имели управляемую кардинальность и поддерживали SLO вместо “пороговых” алертов.

Core Features & Use Cases

  • Three-pillar architecture (metrics/logs/traces): проектирует сквозной контур наблюдаемости с единым сборщиком через OpenTelemetry Collector и унифицированной визуализацией в Grafana.
  • LGTM stack setup (Loki + Grafana + Tempo + Prometheus/Mimir): описывает связку для long-term metrics, лог-агрегации и distributed tracing.
  • Cardinality management + structured logs: задаёт подходы к предотвращению взрыва series и рекомендацию по JSON-логированию с полями вроде dag_id/task_id/level.
  • SLO & error-budget tracking: помогает определить SLO, индикаторы и burn-rate алерты (например, через Pyrra), а также контролировать pipeline freshness/throughput/error rate.
  • Observability as Code (Grafana provisioning): автоматизирует публикацию дашбордов и алертов через Kubernetes-конфиги.
  • Use Case: при запуске новой платформы данных или при разборе production-инцидента, когда “ничего не видно”, но нужно быстро закрыть gaps по метрикам, логам и трассам.

Quick Start

Use this skill to audit your current monitoring and design an LGTM + OpenTelemetry + SLO-based observability plan that prevents high-cardinality issues and enables pipeline freshness, throughput, and error-rate dashboards and alerts.

Frequently Asked Questions about infra-observability-stack-review

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I design an observability stack for a Kubernetes data platform using LGTM and OpenTelemetry?

To design an LGTM observability stack, integrate metrics, logs, and traces via a unified OpenTelemetry Collector, route long-term metrics to Prometheus or Mimir, aggregate logs in Loki, and correlate traces in Tempo.

What is the best way to control Prometheus cardinality when logging structured JSON data?

Control Prometheus cardinality by enforcing structured JSON logging with specific fields like dag_id and task_id, and apply cardinality management rules at the OpenTelemetry Collector to prevent series explosions.

How do I set up SLO and error-budget burn-rate alerting instead of threshold alerts?

Set up SLO and error-budget burn-rate alerting by defining service level indicators for pipeline freshness, throughput, and error rates, then configure burn-rate alerts using tools like Pyrra.

Can I automate Grafana dashboard and alert provisioning for observability as code?

Yes, you can automate Grafana dashboard and alert provisioning by defining observability-as-code configurations and deploying them through Kubernetes manifests to ensure consistent visualization.

Does OpenTelemetry Collector work with Loki and Tempo for incident debugging?

OpenTelemetry Collector works with Loki and Tempo by routing logs and traces to their respective backends, enabling consistent incident debugging and cross-correlation within the Grafana LGTM ecosystem.

When should I audit my observability stack for metrics and trace gaps?

Audit your observability stack when launching a new data platform or during a production incident where visibility is lacking, to quickly identify and close gaps across metrics, logs, and traces.