shine-ab-test-plan

To create an A/B test plan, you define a clear hypothesis, primary metric, and minimum detectable effect (MDE). This allows you to compute the required sample size and recommended test duration to ensure your experiment achieves adequate statistical power.

Experiment guardrails should include predefined decision rules, baseline metrics, analysis methods, and safety mitigations. These guardrails outline specific rollout steps and monitoring criteria across variants to prevent negative impacts during deployment.

A minimum detectable effect (MDE) is the smallest improvement in your primary metric that your A/B test is designed to capture statistically. Defining the MDE is needed to calculate the required sample size and determine the experiment duration.

You determine experiment duration by calculating the required sample size based on your primary metric, MDE, and statistical power. This calculation ensures you gather enough data across all variants to reach a statistically valid conclusion.

Yes, you can apply this A/B test planning approach to both product and marketing experiments. It structures your hypothesis, measurement criteria, and rollout rules to ensure clear evaluation across different variants in either context.