statistical-analysis

To calculate A/B test sample size, you need to define control versus treatment groups, select guardrail and primary metrics, and derive sample size requirements using statistical power and effect size assumptions to avoid validity pitfalls.

Hypothesis testing evaluates null and alternative hypotheses using p-values and effect sizes, while multiple-comparison correction methods adjust statistical conclusions to control error types when testing numerous metrics simultaneously.

Time series forecasting assesses data stationarity and decomposes metrics into trend, seasonality, and noise components, applying methods like ARIMA or exponential smoothing to generate predictions for business tasks.

Descriptive statistics summarize raw experimental and observational data by computing central tendency, dispersion, correlations, and confidence intervals to translate metrics into actionable statistical conclusions.

You must check stationarity before time series forecasting because non-stationary data destabilizes models; decomposing trend, seasonality, and noise ensures your ARIMA or exponential smoothing predictions remain valid.

Multiple testing increases error rates because evaluating numerous hypotheses simultaneously raises the probability of false positives, requiring correction methods to maintain valid statistical conclusions across experiments.