ab-test-setup

To calculate A/B test sample size and duration, define your primary metrics, expected effect size, and statistical significance thresholds. This Skill enforces predefined test duration and sample size calculations to prevent premature peeking and ensure valid growth experimentation results.

The best way to structure growth experimentation hypotheses uses the framework: Because [observation], we believe [change] will cause [outcome] for [audience]. This ensures clear A/B test hypotheses linking changes to expected outcomes and predefined success metrics.

You can run two-variant A/B tests, multi-variant A/B/n experiments, and multivariate testing (MVT). The framework provides test design guidance across pages, features, or copy, ensuring statistically valid experimentation across multiple variants.

You need guardrail metrics in A/B testing to protect against unintended negative consequences while tracking primary and secondary success metrics. This framework enforces primary, secondary, and guardrail metrics to guide product decisions with confidence and prevent harmful outcomes.

You should use an A/B test framework when making data-driven product decisions to reduce guesswork and accelerate learning. It provides reusable playbooks and templates for planning, execution, and results documentation, ensuring statistically valid growth experiments rather than relying on intuition.