skills.rest
marmikcfc
marmikcfcCOMMUNITY
Explore all skills by this author →
GitHub

experiment-monitor

Detect significant results and guardrail breaches in active experiments daily.

Updated Apr 19, 2026
Search Tags:#monitoring#statistics#alerts#archiving#guardrails#cron#experiment
One-click install
npx skills add https://github.com/marmikcfc/pepper-skills --skill experiment-monitor-marmikcfc
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: experiment-monitor
Source: https://github.com/marmikcfc/pepper-skills/tree/main/skills/experiment-monitor
Command: npx skills add https://github.com/marmikcfc/pepper-skills --skill experiment-monitor-marmikcfc

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Automatically monitors all active experiments to detect when they reach significance, breach guardrails, or become stale, enabling timely founder alerts and archival actions.

Core Features & Use Cases

  • Daily automatic evaluation of experiments in experiments/active/ against their metrics in metrics/daily/.
  • Significance checks (p-value) to surface statistically meaningful results.
  • Guardrail breach detection with immediate alerts and suggested archiving.
  • Automated workflow for completing experiments by triggering experiment-archive and generating concise summaries for leadership.

Quick Start

Run this skill on the daily cron to keep experiments healthy and up-to-date.

Frequently Asked Questions about experiment-monitor

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate daily experiment health checks and alerts?▼

Daily experiment monitoring evaluates active experiments against metrics/daily/ to detect statistical significance and guardrail breaches. It runs on a daily cron, generates alerts, and triggers experiment archival when completed.

How do I detect statistically significant results in active experiments?▼

Detect statistically significant results by running daily p-value checks on active experiments. The monitoring process reads metrics/daily/ data to surface meaningful results and writes concise summaries to metrics/alerts.md for leadership review.

Do I need PEPPER_CLOUD_URL and PEPPER_API_KEY to monitor experiments?▼

Yes, PEPPER_CLOUD_URL and PEPPER_API_KEY are enforced prerequisites for the experiment monitoring skill. These environment variables must be configured before the daily cron can evaluate active experiments and trigger archival.

Can I automatically trigger archival when an experiment breaches guardrails?▼

Yes, you can automatically trigger archival when guardrails are breached. The monitoring process detects guardrail breaches, issues immediate alerts, and triggers experiment-archive to complete the experiment workflow.

What is the best way to alert founders when experiments become stale?▼

The best way to alert founders about stale experiments is through daily cron monitoring. It identifies stale experiments alongside significance and guardrail breaches, writing timely alerts to metrics/alerts.md.

Related Skills

Explore More Data & Analytics Skills→
marmikcfcmarmikcfc

experiment-preregister

Locks an experiment spec with a cryptographic hash before data collection.

Community
Intermediate
fitlab-aifitlab-ai
84

complete-task

Validate prerequisites, update metadata, and archive completed tasks.

Official
Advanced
OneWave-AIOneWave-AI
245

intent-signal-aggregator

Aggregate buyer-intent signals from job postings, funding rounds, and leadership changes.

Official
Advanced
omeragaakbasomeragaakbas

ab-test-analysis

Analyze A/B test results to compute statistical significance and practical impact.

Community
Intermediate