ab-test-setup

Design and analyze A/B tests with hypotheses, sample sizes, and metrics.

Updated Apr 14, 2026
One-click install
npx skills add https://github.com/dotruru/claudemd --skill ab-test-setup-dotruru
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: ab-test-setup
Source: https://github.com/dotruru/claudemd/tree/main/skills/ab-test-setup
Command: npx skills add https://github.com/dotruru/claudemd --skill ab-test-setup-dotruru

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Plan and execute statistically rigorous experiments to identify changes that drive meaningful improvements, reducing guesswork and bias in decision making.

Core Features & Use Cases

  • Hypothesis-driven test planning to formalize ideas into testable hypotheses with measurable outcomes.
  • Flexible test types including A/B tests, A/B/n, multivariate tests, and split URL experiments across product, marketing, and pricing pages.
  • Metrics, sample size, and guardrails guidance to ensure valid conclusions and robust decision-making.
  • Learning and documentation to capture results, insights, and next steps for a reusable testing repository.

Quick Start

Define your hypothesis, select your primary metric, estimate baseline performance, and compute the required sample size before starting the test.

Frequently Asked Questions about ab-test-setup

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I calculate sample size for an A/B test?

To calculate A/B test sample size, define your hypothesis, select a primary metric, estimate baseline performance, and perform a power calculation to ensure statistically valid conclusions. This framework reduces guesswork by establishing robust guardrails before testing begins.

What is the difference between A/B/n testing and multivariate tests?

A/B/n testing compares multiple distinct variants against a control to find a single winning change, while multivariate tests examine combinations of elements to identify interactions. Selecting the right experiment design ensures reliable outcomes for product or pricing changes.

How do I formulate a testable hypothesis for product experimentation?

Formulating an A/B test hypothesis requires formalizing your idea into a clear statement with measurable outcomes and a primary metric. This hypothesis-driven test planning ensures structured learning and formalizes ideas into rigorous experiments.

Can I use A/B testing for pricing and marketing page experiments?

Yes, A/B testing can be applied across website product features, pricing models, and marketing experiments. The framework supports flexible test types including split URL experiments to determine which changes drive meaningful improvements.

What guardrails do I need for valid A/B test conclusions?

Establishing A/B test guardrails involves defining clear metrics, computing required sample sizes, and performing power calculations. These precautions prevent bias in decision making and ensure your test results provide reliable, structured learning.