skills.rest
PostHog
PostHogOFFICIAL
Explore all skills by this author →
Website𝕏@PostHogGitHub

monitoring-capture-service

Diagnose PostHog capture service health using Grafana MCP metrics, logs, and profiles.

713|118|Updated Aug 11, 2020
Search Tags:#Rust#grafana#prometheus#Kafka#incident diagnosis#Monitoring#Observability
One-click install
npx skills add https://github.com/PostHog/posthog-foss --skill monitoring-capture-service
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: monitoring-capture-service
Source: https://github.com/PostHog/posthog-foss/tree/main/.agents/skills/monitoring-capture-service
Command: npx skills add https://github.com/PostHog/posthog-foss --skill monitoring-capture-service

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Investigating production issues in PostHog's Rust capture service requires knowing hundreds of metric names, label scopes, Kafka topics, Redis instances, and dashboard UIDs across two environments. This Skill provides the operational knowledge and discovery workflows to diagnose latency, event loss, Kafka backpressure, rate limiting, and Redis issues without memorizing unstable metric names.

Core Features & Use Cases

  • Live metric discovery: Enumerate capture, Kafka, Envoy, and MSK metrics via Grafana MCP tools scoped by namespace, container, and environment.
  • Investigation playbooks: Step-by-step workflows for health checks, event loss funnels, latency diagnosis, Kafka backpressure, rate limiting, and Redis failures.
  • Environment topology: Reference tables for five capture deployments, Envoy cluster naming, Redis instance roles, Kafka topic partition counts, and Pyroscope services in prod-us and prod-eu.
  • Use Case: When events appear to be dropping in production, follow the event lifecycle funnel playbook to compare received versus ingested rates, enumerate drop causes, and isolate whether quota limits, event restrictions, or Kafka delivery errors are responsible.

Quick Start

Ask the AI to check whether the capture service is healthy in prod-us using the Grafana MCP dashboards and metrics.

Frequently Asked Questions about monitoring-capture-service

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I check if the PostHog capture service is healthy?▼

Start with the Grafana dashboard UID "capture" for an overview, then verify HTTP error rates, Kafka broker connectivity via capture_kafka_any_brokers_down, the ingested-to-received event ratio, and Envoy backend membership health scoped by namespace.

How do I investigate event loss in the capture pipeline?▼

Compare rates of capture_events_received_total against capture_events_ingested_total, then enumerate drop causes on capture_events_dropped_total by its cause label. Check Kafka produce errors, quota limit exceedances, and event restriction drops to isolate the source.

Does this monitoring approach work for both prod-us and prod-eu?▼

Yes, but each Grafana MCP session connects to a single environment's Grafana instance, so cross-environment comparison requires switching instances. MSK and CloudWatch metrics carry environment labels and can be compared within one query.

How do I diagnose Kafka backpressure from the capture service?▼

Check the producer queue depth against its limit, inspect per-broker produce RTT at p99, and verify broker connectivity metrics. Then examine MSK throttle time and network processor idle percentage to identify broker-side saturation.

Why are capture events being rate limited or rerouted?▼

Multiple limiters can apply: billing quotas per team, event restrictions synced from Redis, the global per-token rate limiter, and Contour per-IP limits at the Envoy layer. Check capture_quota_limit_exceeded, capture_event_restrictions_applied, and capture_events_rerouted_overflow by reason.

What are the limitations of capture Redis monitoring?▼

The capture binary emits no direct Redis latency metrics, so Redis health is inferred from capture-side error and staleness gauges plus CloudWatch ElastiCache metrics. Capture also fails open on most Redis failures, so outages may not block traffic.

Related Skills

Explore More Software Engineering Skills→
zocomputerzocomputer
45

grafana-setup

Install and configure Grafana with a Loki datasource for log visualization.

Official
Intermediate
redpanda-dataredpanda-data
6

connect-cdc-spanner

Stream Google Cloud Spanner CDC mutations into Redpanda or Kafka.

Official
Advanced
ntk148vntk148v
42

altinity-expert-clickhouse-expert

Diagnose ClickHouse server health issues by analyzing logs, metrics, and query patterns.

Community
Advanced
vibercoder79vibercoder79
8

grafana

Generate and update Grafana Cloud dashboards, panels, and PromQL alert rules.

Community
Intermediate