measure-experiment-design

Design end-to-end experiment plans with hypotheses, variants, metrics, and guardrails.

527|69|Updated Jan 9, 2026

Search Tags:#A/B testing #metrics #data-driven #hypothesis #product-management #sample-size #experiment-design

Ready to run

npx skills add https://github.com/product-on-purpose/pm-skills --skill measure-experiment-design

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: measure-experiment-design
Source: https://github.com/product-on-purpose/pm-skills/tree/main/skills/measure-experiment-design
Command: npx skills add https://github.com/product-on-purpose/pm-skills --skill measure-experiment-design

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Designing experiments is often ad-hoc, leading to unclear hypotheses, undefined success criteria, and inconsistent reporting. This Skill provides a repeatable framework to craft rigorous experiment plans that align teams and data-driven decision-making.

Core Features & Use Cases

Hypothesis-driven design: clearly state hypotheses and success criteria before starting.
End-to-end planning: specify variants, metrics (primary and guardrails), sample size, duration, and targeting.
Artifact templating: produce a standardized experimental design document that can be shared across teams and archived.

Quick Start

Design an experiment plan for onboarding flow with a primary metric of activation rate, MDE of 5 percentage points, and a 14-day duration. Example: design_experiment_plan --name onboarding-variant --start 2026-02-01 --end 2026-02-15 --primary-metric activation_rate --mde 5 --traffic-split 50/50

Frequently Asked Questions about measure-experiment-design

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I design an A/B testing plan for a product feature?▼

To design an A/B testing plan, you define clear product hypotheses, specify control and treatment variants, set primary metrics and guardrails, and calculate the required sample size and test duration. This ensures data-driven validation of feature changes.

What metrics should I include in an experiment design?▼

An experiment design should include a primary metric to measure the main hypothesis, secondary metrics for additional insights, and guardrail metrics to prevent negative impacts on other areas. It also requires defining audience inclusion and exclusion criteria.

How do I calculate sample size and duration for an A/B test?▼

Calculating sample size and duration for an A/B test requires defining the minimum detectable effect (MDE), the primary metric baseline, and the traffic split. The experiment plan standardizes these parameters to ensure statistically valid results.

What is the best way to structure a controlled experiment plan?▼

The best way to structure a controlled experiment plan is using a standardized template that documents hypotheses, variants, metrics, targeting, data collection plans, and risk considerations. This creates a shareable artifact that aligns teams on data-driven decision-making.

Can I use this experiment design framework for onboarding and pricing changes?▼

Yes, you can use this experiment design framework for onboarding, pricing, and feature changes. It applies to controlled experiments across diverse product areas by standardizing hypothesis validation, traffic splitting, and metric tracking.

Why do I need guardrail metrics in an A/B test?▼

You need guardrail metrics in an A/B test to monitor and prevent unintended negative consequences on unrelated product areas. They act as safety thresholds alongside your primary success metrics to ensure overall product health during the experiment.