ab-test-setup

Plan, design, and implement A/B tests with hypothesis, metrics, and sample size.

2|Updated Mar 13, 2026
One-click install
npx skills add https://github.com/zhangzhang-111-i/claude-skills111 --skill ab-test-setup-zhangzhang-111-i
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: ab-test-setup
Source: https://github.com/zhangzhang-111-i/claude-skills111/tree/main/marketing-skill/ab-test-setup
Command: npx skills add https://github.com/zhangzhang-111-i/claude-skills111 --skill ab-test-setup-zhangzhang-111-i

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill helps users plan, design, and implement A/B tests to optimize product features, marketing copy, or user interfaces by providing a structured approach to experimentation.

Core Features & Use Cases

  • Hypothesis Formulation: Guides users to create clear, testable hypotheses using a proven framework.
  • Metric Selection: Advises on choosing primary, secondary, and guardrail metrics for robust analysis.
  • Sample Size & Duration Calculation: Provides tools and guidance for determining the necessary sample size and test duration.
  • Variant Design Best Practices: Offers principles for creating effective test variants.
  • Use Case: A marketing manager wants to test a new headline on a landing page to increase sign-ups. This Skill will help them define the hypothesis, choose metrics, calculate sample size, and outline the variant design.

Quick Start

Use the ab-test-setup skill to plan an A/B test for a new call-to-action button.

Frequently Asked Questions about ab-test-setup

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I design an A/B test for conversion rate optimization?

To design an A/B test for conversion rate optimization, formulate a clear hypothesis, select primary and guardrail metrics, calculate the required sample size, and outline variant design best practices to ensure robust analysis.

What metrics should I track when running marketing campaign experiments?

When running marketing campaign experiments, track primary metrics to measure main goals, secondary metrics for deeper insights, and guardrail metrics to prevent unintended negative impacts on user experience.

How do I calculate sample size and test duration for hypothesis testing?

Calculate sample size and test duration for hypothesis testing by defining your expected effect size, determining statistical power and significance levels, and applying established statistical formulas provided in the skill's references.

What is the best way to structure a hypothesis for landing page experimentation?

The best way to structure a hypothesis for landing page experimentation is using a proven framework that defines the expected change, the specific user behavior impacted, and the measurable outcome for data-driven decisions.

Can I use this approach to test user experience improvements without advanced data analysis knowledge?

Yes, you can test user experience improvements without advanced data analysis knowledge by following the skill's structured guidance on variant design, metric selection, and understanding basic test analysis methodologies.