experiment-monitor

Detect significant results and guardrail breaches in active experiments daily.

Updated Apr 19, 2026

Search Tags:#monitoring #statistics #alerts #archiving #guardrails #cron #experiment

Ready to run

npx skills add https://github.com/marmikcfc/pepper-skills --skill experiment-monitor-marmikcfc

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: experiment-monitor
Source: https://github.com/marmikcfc/pepper-skills/tree/main/skills/experiment-monitor
Command: npx skills add https://github.com/marmikcfc/pepper-skills --skill experiment-monitor-marmikcfc

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Automatically monitors all active experiments to detect when they reach significance, breach guardrails, or become stale, enabling timely founder alerts and archival actions.

Core Features & Use Cases

Daily automatic evaluation of experiments in experiments/active/ against their metrics in metrics/daily/.
Significance checks (p-value) to surface statistically meaningful results.
Guardrail breach detection with immediate alerts and suggested archiving.
Automated workflow for completing experiments by triggering experiment-archive and generating concise summaries for leadership.

Quick Start

Run this skill on the daily cron to keep experiments healthy and up-to-date.

Frequently Asked Questions about experiment-monitor

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I automate daily experiment health checks and alerts?▼

Daily experiment monitoring evaluates active experiments against metrics/daily/ to detect statistical significance and guardrail breaches. It runs on a daily cron, generates alerts, and triggers experiment archival when completed.

How do I detect statistically significant results in active experiments?▼

Detect statistically significant results by running daily p-value checks on active experiments. The monitoring process reads metrics/daily/ data to surface meaningful results and writes concise summaries to metrics/alerts.md for leadership review.

Do I need PEPPER_CLOUD_URL and PEPPER_API_KEY to monitor experiments?▼

Yes, PEPPER_CLOUD_URL and PEPPER_API_KEY are enforced prerequisites for the experiment monitoring skill. These environment variables must be configured before the daily cron can evaluate active experiments and trigger archival.

Can I automatically trigger archival when an experiment breaches guardrails?▼

Yes, you can automatically trigger archival when guardrails are breached. The monitoring process detects guardrail breaches, issues immediate alerts, and triggers experiment-archive to complete the experiment workflow.

What is the best way to alert founders when experiments become stale?▼

The best way to alert founders about stale experiments is through daily cron monitoring. It identifies stale experiments alongside significance and guardrail breaches, writing timely alerts to metrics/alerts.md.