ab-testing

To calculate sample size for an A/B test, you must define your primary metrics, guardrails, and hypothesis first. This skill computes and applies appropriate sample sizes to ensure statistically rigorous results across pages, flows, and features.

A/B/n testing compares multiple variations against a control simultaneously, while multivariate testing evaluates interactions between multiple elements on a page. This skill helps you select the appropriate test type based on your hypothesis and experiment scope.

Guardrail metrics protect your business by monitoring for negative side effects during experiments. You define them alongside primary and secondary metrics, track them throughout the test, and ensure they remain within acceptable limits before acting on results.

The best way to frame an A/B test hypothesis is to clearly state the expected change, the affected metric, and the rationale. This skill provides standardized templates to structure hypotheses that yield credible, actionable experiment insights.

Yes, split URL testing is ideal for comparing fundamentally different page layouts or flows. This skill supports split URL tests alongside A/B, A/B/n, and multivariate tests to validate design changes across your pages and features.

A/B test results lack significance when sample sizes are insufficient or guardrails are breached. This skill computes appropriate sample sizes upfront and provides rigorous measurement frameworks to ensure your outcomes are credible and actionable.