experiment-designer

To calculate A/B test sample size, you input the baseline conversion rate, minimum detectable effect, and statistical power into the Python script to receive the required participant count per variant for rigorous hypothesis testing.

The best way to formulate a product experimentation hypothesis is using the If/Then/Because structure, which this skill guides you to create, ensuring your hypothesis links the proposed change directly to the expected metric outcome.

To prioritize A/B testing ideas with ICE scoring, you evaluate each experiment's Impact, Confidence, and Ease. This skill implements the ICE method to rank your testing backlog, ensuring you execute high-value experiments first.

To define primary and guardrail metrics for an A/B test, you separate the main success indicator from metrics you must protect against regression. This skill assists in defining primary, guardrail, and secondary metrics to ensure balanced experiment measurement.

An A/B test result can show statistical significance without practical impact when the detected effect size is too small to matter for the business. This skill provides guardrails to interpret statistical significance alongside practical impact, preventing common experiment pitfalls.