ab-test-analysis

What problem does it solve?

Analyzing A/B tests and experiments to determine if observed differences are real and actionable, reducing decision risk.

Core Features & Use Cases

• Significance testing: compute p-values, z-scores, and confidence intervals for variant comparisons.
• Power analysis & planning: estimate required sample sizes and achieved power to inform study design.
• Guardrail checks & SRM: detect sample ratio mismatches and evaluate impacts on key secondary metrics.
• Decision recommendations: provide clear ship/don't ship guidance with estimated business impact.
• Use Case: You ran an email campaign test and want to know if open rate changes are significant and whether to roll out to all users.

Quick Start

Analyze a completed A/B test by inputting variant data and design details and request a concise decision report.