skills.rest
sangrokjung
sangrokjungCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

ab-test-setup

Plan and implement A/B tests with hypothesis formulation and sample size calculation.

1|Updated Feb 7, 2026
Search Tags:#hypothesis testing#experimentation#ab testing#variant design#product analytics#conversion rate optimization
One-click install
npx skills add https://github.com/sangrokjung/claude-code-config-public --skill ab-test-setup-sangrokjung
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ab-test-setup
Source: https://github.com/sangrokjung/claude-code-config-public/tree/main/commands/ab-test-setup
Command: npx skills add https://github.com/sangrokjung/claude-code-config-public --skill ab-test-setup-sangrokjung

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill helps users plan, design, and implement A/B tests and experiments to improve user experience and achieve business goals through data-driven decisions.

Core Features & Use Cases

  • Hypothesis Formulation: Guides users to create strong, testable hypotheses using a clear framework.
  • Test Design: Provides principles and methods for choosing test types (A/B, MVT), calculating sample sizes, and selecting metrics.
  • Variant Creation: Offers best practices for designing effective control and variant experiences.
  • Use Case: A product manager wants to test a new headline on a landing page to increase sign-ups. This Skill will help them formulate a hypothesis, determine the necessary sample size, and define primary and secondary metrics.

Quick Start

Help me design an A/B test for a new signup button color.

Frequently Asked Questions about ab-test-setup

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I formulate a strong hypothesis for A/B testing?▼

A strong A/B testing hypothesis uses a clear framework to define the expected change, the target metric, and the predicted outcome, ensuring your experiment is testable and directly tied to product optimization goals.

What is the best way to calculate sample size for an A/B test?▼

Calculating sample size for an A/B test requires selecting your test type, defining primary metrics, and estimating the baseline conversion rate to ensure statistically significant results for your product or marketing optimization.

When should I use multivariate testing instead of a standard A/B split test?▼

Multivariate testing is used instead of a standard A/B split test when you need to simultaneously evaluate multiple variant design elements and understand their combined impact on conversion rate optimization.

How do I define primary and secondary metrics for product analytics experiments?▼

Defining metrics for product analytics experiments involves selecting a primary metric to measure your hypothesis directly and secondary metrics to monitor unintended effects on the overall user experience.

What are the best practices for traffic allocation in variant design?▼

Traffic allocation in variant design distributes users between control and variant groups, ensuring enough exposure per variant to reach the calculated sample size and achieve reliable experiment results.

Related Skills

Explore More Product & Management Skills→
mrsknetworkmrsknetwork

ab-test-setup

Plan statistically sound A/B tests with sample-size calculations and metric selection.

Community
Intermediate
sanandnarayansanandnarayan
1

ab-test-setup

Plan, design, and implement A/B tests for conversion rate optimization.

Community
Intermediate
osvikosvik
4

experiment

Generate A/B test implementations with the Abrantes JavaScript library.

Community
Advanced
openyakopenyak
704

user-research

Plan, conduct, and synthesize user research studies into actionable insights.

Official
Intermediate