ab-test-setup

Plan A/B, A/B/n, and multivariate experiments with sample-size calculations.

Updated Apr 21, 2026
One-click install
npx skills add https://github.com/scanbott/claude-skills --skill ab-test-setup-scanbott
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: ab-test-setup
Source: https://github.com/scanbott/claude-skills/tree/main/ab-test-setup
Command: npx skills add https://github.com/scanbott/claude-skills --skill ab-test-setup-scanbott

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Designing and executing rigorous growth experiments to determine which changes drive measurable improvements, while avoiding wasted effort on unproven ideas.

Core Features & Use Cases

  • Hypothesis-driven test planning using A/B, A/B/n, MVT, and split URL
  • Statistical rigor including sample-size planning, power analysis, and appropriate duration
  • Metrics guidance (primary/secondary/guardrails) and result interpretation
  • Growth experimentation program guidance and playbook documentation
  • Comprehensive templates for planning, running, and reporting tests (test templates, results, and repositories)

Quick Start

Outline a test idea and craft a formal hypothesis and test plan using the standard A/B framework.

Frequently Asked Questions about ab-test-setup

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I calculate the required sample size for an A/B test?

To calculate sample size for an A/B test, you apply statistical power analysis based on your expected effect size and variance. This skill enforces statistical rigor by computing the required sample size and appropriate test duration before execution.

What is the best way to plan an A/B/n test for a marketing funnel?

The best way to plan an A/B/n test for a marketing funnel is using a hypothesis-driven framework that defines primary, secondary, and guardrail metrics. This approach provides templates to structure multi-variant experiments and document resulting winning patterns.

When do I need to use multivariate experiments instead of standard A/B testing?

You need multivariate experiments instead of standard A/B testing when evaluating multiple variable combinations simultaneously across product pages or onboarding flows. This skill plans MVT designs alongside split URL tests to determine which specific changes drive measurable improvements.

How do I define guardrail metrics for a growth experimentation program?

Defining guardrail metrics for a growth experimentation program involves identifying baseline thresholds that must not degrade during testing. This skill provides metrics guidance to set primary, secondary, and guardrail parameters ensuring experiments remain statistically valid.

Can I use this skill to document and scale winning test patterns?

Yes, you can document and scale winning test patterns using the provided playbook guidance and comprehensive templates. The skill generates standardized repositories for planning, running, and reporting tests to institutionalize successful growth experiments.