ab-testing

To calculate A/B test sample size, you input your baseline conversion rate and expected relative lift to determine the Minimum Detectable Effect (MDE). This translates into feasibility-aware required sample sizes, ensuring your test is properly powered before launch.

A valid CRO hypothesis requires a clear X→Y change tied to an expected metric impact and a specific mechanism. This hypothesis-first design ensures that even a negative result provides actionable information and resists confirmation bias.

Peeking at A/B test results before reaching the required sample size inflates false positives by triggering premature statistical significance. You prevent it by applying fixed-horizon or sequential testing decision rules with predefined stop and ship guidelines.

Ecommerce A/B testing stop, ship, or rollback rules are determined by selecting either fixed-horizon or sequential methodology. These rules define explicit thresholds for measurable metrics, producing actionable guidance to prevent invalid conclusions.

Running multiple CRO experiments simultaneously compromises test isolation and invalidates results. Proper experimentation enforces one-variable-per-test isolation, supporting accurate cadence planning and reliable stop, ship, or rollback decisions.

You should not use sequential testing for A/B experiments when you lack the infrastructure to continuously monitor statistical boundaries or when strict one-variable-per-test isolation is not maintained, as this leads to peeking-induced false positives.