signals-scout-anomaly-detection

Detects anomalies in PostHog dashboards and insights against seasonality-matched baselines.

713|118|Updated Aug 11, 2020
One-click install
npx skills add https://github.com/PostHog/posthog-foss --skill signals-scout-anomaly-detection
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: signals-scout-anomaly-detection
Source: https://github.com/PostHog/posthog-foss/tree/main/products/signals/skills/signals-scout-anomaly-detection
Command: npx skills add https://github.com/PostHog/posthog-foss --skill signals-scout-anomaly-detection

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

Teams miss sudden spikes, drops, flat-lines, and trend breaks in their most-viewed PostHog dashboards and insights until long after they happen, and naive alerting drowns them in seasonal false positives.

Core Features & Use Cases

  • Seasonality-aware anomaly scoring: Scores the latest complete bucket of saved time-series insights using PostHog's alert-simulate detectors (z-score, MAD, isolation forest, ensembles) or a robust MAD-based z-score fallback via HogQL.
  • Durable watchlist with explore/exploit cadence: Maintains a scratchpad-backed watchlist of high-value insights with baselines, cadences, and round-robin scheduling so coverage compounds across runs.
  • Report authoring with notebook evidence: Files deduplicated inbox reports with quantified evidence, segment attribution, priority, and a linked notebook write-up containing embedded charts.
  • Use Case: A daily signups metric drops 60% versus its same-weekday baseline; the scout scores it (robust z = 4.8), attributes the move to a segment, and authors a P1 inbox report with a notebook showing the baseline math and charts.

Quick Start

Ask the scout to check the team's most-viewed PostHog dashboards and insights for recent anomalies and report anything that clears its seasonality-matched baseline.

Frequently Asked Questions about signals-scout-anomaly-detection

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I detect anomalies in PostHog insights automatically?

Use the alert-simulate tool to run PostHog's production anomaly detectors (z-score, MAD, isolation forest, or ensembles) on a saved time-series insight. It returns per-point anomaly scores and triggered dates without requiring an alert to exist.

How does the scout avoid seasonal false positives in metrics?

It scores each metric against a seasonality-matched baseline: same day-of-week over 6-8 weeks for daily metrics, or same hour-of-day over 2-4 weeks for hourly ones. It also excludes the current partial bucket and enforces minimum relative-change and absolute-count floors.

Can alert-simulate score SQL-based PostHog insights?

No, alert-simulate only accepts TrendsQuery insights and rejects DataVisualizationNode insights wrapping HogQL queries. For those, the scout falls back to a hand-computed robust z-score over a series built with execute-sql.

What happens when the scout finds a real anomaly?

It authors an inbox report via scout-emit-report with a quantified title, evidence, priority, and suggested reviewers, after first building a notebook with embedded charts and baseline math. Recurrences update the existing report via edit_report instead of filing duplicates.

Why does the scout skip low-count or flat metric series?

Low-count series produce large percentage swings from a few events, so a minimum baseline of roughly 20 events per bucket and a 50% relative-move floor are enforced. Flat series with zero MAD fall back to absolute-change checks to avoid infinite z-scores.