ab-test-setup

Design and run statistically valid A/B tests with hypothesis frameworks and metric definitions.

Updated Apr 8, 2026
One-click install
npx skills add https://github.com/SlevoDev/s-tag --skill ab-test-setup-slevodev
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ab-test-setup
Source: https://github.com/SlevoDev/s-tag/tree/main/.claude/skills/ab-test-setup
Command: npx skills add https://github.com/SlevoDev/s-tag --skill ab-test-setup-slevodev

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Plan, design, and execute statistically valid A/B tests and growth experiments to determine what drives measurable improvements.

Core Features & Use Cases

  • Hypothesis-driven test planning across A/B, A/B/n, MVT, and split URL tests.
  • Metrics framework: define primary, secondary, and guardrail metrics; estimate sample size and duration.
  • Reusable templates and playbooks for test plans, results, backlog, and playbooks to drive ongoing experimentation.

Quick Start

Create a hypothesis-driven test plan with clearly defined metrics, sample size, duration, and variant mapping for a page or feature.

Frequently Asked Questions about ab-test-setup

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I set up an A/B test with statistical significance?▼

To set up an A/B test with statistical significance, define a clear hypothesis, establish primary, secondary, and guardrail metrics, and calculate the required sample size and test duration to ensure reliable decision making.

What is the best way to plan growth experiments and define test metrics?▼

The best way to plan growth experiments is using a hypothesis-driven framework that maps variant designs to specific metric definitions, ensuring your primary, secondary, and guardrail metrics are established before execution begins.

How do I calculate sample size and duration for split testing?▼

Calculating sample size and duration for split testing involves estimating the required audience size based on your expected effect size and metric variance, ensuring the test runs long enough to achieve statistical significance.

Can I use this approach for multivariate tests and A/B/n experiments?▼

Yes, this approach applies when planning, designing, or implementing A/B tests, split tests, A/B/n experiments, and multivariate tests to compare two or more approaches effectively.

What are guardrail metrics and why do I need them for experimentation?▼

Guardrail metrics are predefined measurements used to monitor and prevent unintended negative consequences during experimentation, ensuring that optimizing primary metrics does not harm overall system health or user experience.

Does this method provide templates for A/B test results and backlogs?▼

Yes, this method provides reusable templates and playbooks for test plans, results tracking, backlog management, and ongoing experimentation to drive continuous growth learning.