statistical-analyst

Hypothesis testing on A/B test results applies frequentist tests to your control and treatment data, computing p-values and confidence intervals to determine whether observed differences are statistically significant rather than random noise.

Calculating required sample size before launch involves verifying power assumptions and computing group sizes needed to detect a meaningful effect, ensuring your experiment is robust enough to yield statistically significant results.

Interpreting existing reported metrics evaluates already-collected proportions, means, or categories, translating statistics into actionable business decisions by applying hypothesis tests and computing confidence intervals post-hoc.

Analyzing both conversion proportions and continuous means is supported, applying frequentist hypothesis tests across different data types to compute p-values, confidence intervals, and effect sizes for comprehensive post-hoc analysis.

A/B test statistical significance without practical impact occurs when p-values are low but effect sizes like Cohen's d or h are negligible, meaning the detected difference is real but too small to justify business decisions.