What problem does it solve?
This Skill helps teams plan, design, and execute statistically valid A/B tests to understand impact on key metrics.
Core Features & Use Cases
- Hypothesis-driven planning: Frame clear hypotheses to isolate changes and learn quickly.
- Sample size & power planning: Calculate required sample sizes and timelines to achieve reliable results.
- Variant design & documentation: Define and document control and variant details, expected outcomes, and success criteria.
- Results interpretation: Guide decision-making with significance, practical impact, and segment insights.
- Use Case: A marketing page test to compare two headlines and two CTAs, with primary metric defined and a plan for data collection and decision rules.
Quick Start
Design a test plan for the homepage CTA. Define the hypothesis, primary metric, target sample size, test duration, and two variants (A and B). Then specify how success will be measured and what constitutes a winner.