skills.rest
stephanesoares
stephanesoaresCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

ab-test-setup

Plan and execute A/B tests with hypotheses, metrics, and sample-size estimation.

Updated Mar 5, 2026
Search Tags:#analysis#metrics#statistics#hypothesis#experiments#ab-testing#sample-size
One-click install
npx skills add https://github.com/stephanesoares/dotfiles --skill ab-test-setup-stephanesoares
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ab-test-setup
Source: https://github.com/stephanesoares/dotfiles/tree/main/config/claude/skills/ab-test-setup
Command: npx skills add https://github.com/stephanesoares/dotfiles --skill ab-test-setup-stephanesoares

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill helps teams design and implement rigorous A/B tests and experiments to validate changes before full rollout.

Core Features & Use Cases

  • Framework for forming test hypotheses and selecting metrics
  • Guidance on sample size planning, test types (A/B, MVT), and result interpretation
  • Use cases across product, marketing, pricing, and UX optimization

Quick Start

Run a quick test by defining a hypothesis, selecting a primary metric, creating two variants, and starting data collection.

Frequently Asked Questions about ab-test-setup

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I set up an A/B test with a structured hypothesis and metrics?▼

To set up an A/B test, define a structured hypothesis, select primary and secondary metrics, establish guardrails, design variants, and estimate the required sample size before data collection.

What is the right sample size for my A/B experiment?▼

Sample size estimation for A/B experiments depends on your primary metric's baseline, minimum detectable effect, and statistical power. Calculating this ensures reliable hypothesis validation before rollout.

Can I use this A/B testing framework for marketing and UX optimization?▼

Yes, this A/B testing framework applies to product, marketing, pricing, and UX optimization experiments across web or app features to validate changes before full rollout.

What's the best way to design multivariate tests alongside standard A/B tests?▼

For multivariate testing (MVT) alongside A/B tests, define multiple variant combinations, select appropriate primary and guardrail metrics, and ensure your sample size supports the expanded test matrix.

How do I interpret A/B test results and document the analysis?▼

Interpret A/B test results by analyzing primary and secondary metrics against guardrails, validating the structured hypothesis, and documenting the findings in a repeatable analysis workflow.

Related Skills

Explore More Data & Analytics Skills→
BellaBeBellaBe
37

validation-action

Define acceptance criteria and generate validation plans with measurable metrics.

Community
Intermediate
w95w95
145

statistical-analysis

Apply descriptive statistics, trend analysis, outlier detection, and hypothesis testing to datasets.

Community
Intermediate
abdullah1854abdullah1854
1

sub-agent-invocation

Route tasks to specialized sub-agents with context isolation for parallel work.

Community
Advanced
rajitsaharajitsaha
10

ab-test-setup

Plan A/B tests with sample size calculations and success metrics.

Community
Intermediate