ab-test-analysis

Analyze A/B test CSV data to compute significance and recommend next steps.

Updated Apr 8, 2026
One-click install
npx skills add https://github.com/martiraste-lgtm/claude-skills --skill ab-test-analysis-martiraste-lgtm
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: ab-test-analysis
Source: https://github.com/martiraste-lgtm/claude-skills/tree/main/pm-data-analytics-ab-test-analysis
Command: npx skills add https://github.com/martiraste-lgtm/claude-skills --skill ab-test-analysis-martiraste-lgtm

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill helps teams analyze A/B test results with statistical rigor, determine significance, assess sample size adequacy, and translate findings into actionable product decisions.

Core Features & Use Cases

  • Evaluate experiment setup: sample size adequacy, test duration, randomization integrity, and wash-out effects.
  • Compute key statistics: conversion rates for control and variant, relative lift, p-value, and 95% confidence interval.
  • Guardrail checks and decision guidance: interpret results with ship/extend/stop/investigate recommendations.
  • Generate a concise analysis summary with actionable next steps for stakeholders.

Quick Start

Provide your A/B test data or paste a CSV with control and variant results, and I will compute significance and recommend next steps.

Frequently Asked Questions about ab-test-analysis

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I calculate A/B test statistical significance from a CSV export?

To calculate A/B test statistical significance from a CSV export, the tool processes raw control and variant data to compute p-values, conversion rates, relative lift, and 95% confidence intervals for accurate product decisions.

How do I know if my sample size is large enough for A/B testing?

You can determine if your sample size is adequate for A/B testing by running a power check. The analysis evaluates sample size adequacy, test duration, and randomization integrity to ensure your experiment results are valid.

Can I analyze A/B test results with unequal traffic splits?

Yes, you can analyze A/B test results with unequal traffic splits. The evaluation accommodates varying durations and traffic allocations across control and variant groups to accurately calculate conversion rates and statistical significance.

What is the best way to interpret p-values and confidence intervals for product experiments?

The best way to interpret p-values and confidence intervals for product experiments is through structured guardrail checks. This approach translates statistical metrics into actionable ship, extend, stop, or investigate recommendations for stakeholders.

How do I evaluate wash-out effects and randomization integrity in experiment design?

To evaluate wash-out effects and randomization integrity in experiment design, the evaluation assesses your test setup. It checks for sample size adequacy and structural biases to validate the experiment environment before computing final statistics.