What problem does it solve? Designing A/B tests without statistical rigor leads to false positives, wasted traffic, and wrong product decisions. This Skill guides you through hypothesis formulation, sample size calculation, variant design, and result analysis so your experiments produce trustworthy, actionable outcomes. ## Core Features & Use Cases - Hypothesis Framework: Structure test ideas using the observation-change-effect-audience-metric format to ensure every test has a clear, measurable prediction. - Sample Size & Duration Planning: Calculate required traffic per variant using baseline conversion rate, minimum detectable effect, and statistical power, with included Python scripts for programmatic calculation. - Test Execution & Analysis Guidance: Follow pre-launch checklists, avoid peeking pitfalls, select primary/secondary/guardrail metrics, and interpret results with confidence intervals and segment analysis. - Use Case: You want to test a new headline on your pricing page. The Skill helps you write a strong hypothesis, determine you need 12,000 visitors per variant at a 3% baseline, set up a 50/50 split in PostHog, and analyze the Chi-Squared significance of the results. ## Quick Start Ask the AI to help you plan an A/B test for your landing page, including the hypothesis, required sample size, and metrics to track.
