ab-test-analysis

Analyze A/B test results by computing primary metric conversion rates, p-values, and confidence intervals for control and variant groups. Provide experiment duration, traffic splits, and guardrails to receive clear statistics and practical ship, extend, or stop recommendations.

Decide whether to ship an A/B test variant by evaluating statistical significance alongside guardrail metrics. This calculates p-values and confidence intervals to produce practical recommendations on whether to ship, extend, or stop the experiment.

Yes, you can calculate p-values and confidence intervals from raw experiment data by optionally generating Python code. Provide raw data alongside traffic splits and primary metrics to compute statistics and guide product decisions programmatically.

Conversion rates and guardrails affect A/B testing decisions by establishing practical significance and safety thresholds. Analyzing primary metric conversion rates against guardrails ensures statistically significant changes do not negatively impact other critical product areas.

To run an A/B test analysis for web product features, you need the experiment's primary metric, duration, traffic splits, and guardrails. Supplying these inputs computes statistics like p-values and generates actionable product recommendations.