Observability Stack

Implement Prometheus, Grafana, Jaeger, and OpenTelemetry for distributed system observability.

1|Updated Dec 18, 2024
One-click install
npx skills add https://github.com/altrupets/monorepo --skill observability-stack-altrupets
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: Observability Stack
Source: https://github.com/altrupets/monorepo/tree/main/skills/system-reliability-engineering/observability-stack
Command: npx skills add https://github.com/altrupets/monorepo --skill observability-stack-altrupets

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires opentelemetry, opentelemetry-otlp, opentelemetry_sdk, opentelemetry_semantic_conventions, tracing, tracing-opentelemetry, tracing-subscriber, tokio, @opentelemetry/api, @opentelemetry/sdk-node, @opentelemetry/auto-instrumentations-node, @opentelemetry/exporter-otlp-http, @opentelemetry/resources, @opentelemetry/semantic-conventions, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill addresses the challenge of understanding complex distributed systems by providing a comprehensive framework for monitoring their internal state and behavior.

Core Features & Use Cases

  • Metrics Collection & Visualization: Implement Prometheus for collecting time-series metrics and Grafana for creating insightful dashboards and alerts.
  • Distributed Tracing: Set up Jaeger and OpenTelemetry to trace requests across microservices, enabling root cause analysis of performance bottlenecks and errors.
  • Structured Logging: Integrate Loki for efficient log aggregation and querying, correlating logs with traces and metrics.
  • Use Case: Debugging a slow API request that spans multiple microservices by tracing its path, identifying the slowest service, and examining its logs and metrics for issues.

Quick Start

Implement a full observability stack using Prometheus, Grafana, Jaeger, and OpenTelemetry.

Frequently Asked Questions about Observability Stack

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I set up distributed tracing for microservices using OpenTelemetry?

Set up distributed tracing for microservices by integrating OpenTelemetry for instrumentation and Jaeger to trace requests across services, enabling root cause analysis of performance bottlenecks. It supports Rust and Node.js applications.

What is the best way to collect metrics and visualize them with Prometheus and Grafana?

Collect metrics and visualize them by implementing Prometheus for gathering time-series data and Grafana for creating insightful dashboards and alerts. This provides a comprehensive framework for monitoring distributed system states.

Does this observability stack support Rust and Node.js applications?

Yes, the observability stack explicitly supports Rust and Node.js applications. It uses opentelemetry, tracing, and tokio dependencies for Rust, alongside @opentelemetry/sdk-node and auto-instrumentations-node for Node.js environments.

How do I correlate structured logs with traces and metrics in a distributed system?

Correlate structured logs with traces and metrics by integrating Loki for efficient log aggregation and querying. This allows you to examine logs alongside traces to identify issues when debugging slow API requests spanning multiple microservices.

Can I deploy this monitoring stack with Kubernetes configurations?

Yes, you can deploy this monitoring stack using Kubernetes. The implementation provides Kubernetes deployment configurations to set up Prometheus, Grafana, Jaeger, and OpenTelemetry for your distributed systems environment.

Why do I need OpenTelemetry for debugging slow API requests across microservices?

You need OpenTelemetry to debug slow API requests by tracing the exact path across microservices, identifying the slowest service, and examining its correlated logs and metrics to find the underlying performance bottleneck.