ab-test-setup

Plan statistically valid A/B tests with sample-size calculations and variant design.

Updated Jan 13, 2026
One-click install
npx skills add https://github.com/MrGKanev/agent-skills --skill ab-test-setup-mrgkanev
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: ab-test-setup
Source: https://github.com/MrGKanev/agent-skills/tree/main/skills/ab-test-setup
Command: npx skills add https://github.com/MrGKanev/agent-skills --skill ab-test-setup-mrgkanev

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill helps teams plan, design, and execute statistically valid A/B tests to understand impact on key metrics.

Core Features & Use Cases

  • Hypothesis-driven planning: Frame clear hypotheses to isolate changes and learn quickly.
  • Sample size & power planning: Calculate required sample sizes and timelines to achieve reliable results.
  • Variant design & documentation: Define and document control and variant details, expected outcomes, and success criteria.
  • Results interpretation: Guide decision-making with significance, practical impact, and segment insights.
  • Use Case: A marketing page test to compare two headlines and two CTAs, with primary metric defined and a plan for data collection and decision rules.

Quick Start

Design a test plan for the homepage CTA. Define the hypothesis, primary metric, target sample size, test duration, and two variants (A and B). Then specify how success will be measured and what constitutes a winner.

Frequently Asked Questions about ab-test-setup

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I calculate sample size for an A/B test?

To calculate sample size for an A/B test, you frame a clear hypothesis and define your primary metric. This allows you to compute the required sample size and timeline needed to achieve statistically reliable results.

What is the best way to design a hypothesis for web page experiments?

The best way to design a hypothesis for web page experiments is to frame it clearly to isolate specific changes. This approach ensures you quickly learn the impact of variations like different headlines or CTAs on your product metrics.

How do I interpret A/B test results for pricing experiments?

To interpret A/B test results for pricing experiments, you evaluate statistical significance, practical impact, and segment insights. This guides your decision-making by defining whether a variant meets the predefined success criteria.

Can I use this approach for onboarding flows and feature rollouts?

Yes, you can use this A/B testing approach for onboarding flows and feature rollouts. It applies statistical planning to determine the exact impact of changes across various product surfaces and user experiences.

What do I need to document when planning variant designs?

When planning variant designs, you need to document the control and variant details, expected outcomes, and success criteria. This documentation includes the run plan and specifies how the winning variant will be measured.