ab-testing

To calculate A/B test sample size and duration, you must define your primary metric, establish a hypothesis, and apply statistical formulas to ensure valid conversion results. This skill provides structured templates to compute and document these exact parameters before launching experiments.

The peeking problem in A/B testing occurs when you check intermediate results before reaching the required sample size, leading to false positives. You avoid it by setting a fixed test duration upfront and using this skill's structured templates to document results only after completion.

You should use A/B/n testing when comparing more than two variants simultaneously, and multivariate testing (MVT) when testing combinations of multiple elements. This skill guides the selection of appropriate test types based on your specific growth and product experimentation needs.

You create a hypothesis-driven testing framework by defining a clear hypothesis, identifying primary and secondary metrics, and designing variants to test specific growth assumptions. This skill provides structured templates for planning, running, and documenting these experiments to boost product outcomes.

The best metrics to track for A/B testing are a clearly defined primary metric tied to your hypothesis and relevant secondary metrics. This skill supports selecting these metrics to measure performance differences accurately between two approaches in marketing or product scenarios.

You do not need advanced statistics knowledge to run rigorous A/B tests, as this skill guides you through hypothesis creation, sample size calculation, and interpretation. It applies structured templates to ensure statistically valid product and marketing experimentation without requiring deep manual calculations.