create-experiment-design

To design an A/B test with a pre-registered analysis plan, you provide a falsifiable test hypothesis and the skill generates a complete experiment doc with success criteria, statistical tests, alpha/beta handling, and stopping rules.

A pre-registered analysis plan for hypothesis validation defines statistical tests, alpha/beta error handling, and stopping rules before running the experiment, ensuring scientifically valid product decisions and preventing data dredging.

You calculate sample size and duration for an experiment by defining your hypothesis and guardrail metrics, which allows the skill to output practical duration estimates and sample size calculations aligned with your product context.

Yes, you can use guardrail metrics to protect product quality during user experiments by defining them alongside your success criteria, which helps the design avoid harmful trade-offs and maintain product standards.

Yes, this experiment design approach works for both web and mobile app feature experiments, applying falsifiable hypotheses and rigorous A/B testing to validate product changes across different platforms.

The best way to document statistical guardrails and stopping rules for an A/B test is generating an end-to-end experiment document that pre-registers your analysis plan, metrics, and sample size to ensure valid decisions.