skills.rest
Veloxia-agency
Veloxia-agencyCOMMUNITY
Explore all skills by this author →
GitHub

ab-test-setup

Design statistically valid A/B tests with hypotheses, metrics, and sample sizes.

Updated Apr 24, 2026
Search Tags:#marketing analytics#conversion optimization#hypothesis#experimentation#a/b testing#sample size#split testing
One-click install
npx skills add https://github.com/Veloxia-agency/VELOXIA-WEB --skill ab-test-setup-veloxia-agency
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ab-test-setup
Source: https://github.com/Veloxia-agency/VELOXIA-WEB/tree/main/.claude/skills/marketing-skill/skills/ab-test-setup
Command: npx skills add https://github.com/Veloxia-agency/VELOXIA-WEB --skill ab-test-setup-veloxia-agency

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill helps teams design, launch, and evaluate A/B tests without relying on guesswork, so they can make conversion decisions with statistical confidence.

Core Features & Use Cases

  • Hypothesis Design: Turn vague ideas into a testable experiment hypothesis with a clear expected outcome.
  • Planning & Rigor: Define the primary metric, guardrails, traffic split, sample size, and test duration before launch.
  • Analysis & Decisioning: Interpret results, check statistical significance, review segment behavior, and decide whether to ship, reject, or rerun.
  • Use Case: Compare two pricing-page headlines, measure signup-start rate as the primary metric, and determine whether the new version is a real improvement or just random noise.

Quick Start

Ask the skill to design an A/B test for your current change, including the hypothesis, variants, metrics, sample size, and launch checklist.

Frequently Asked Questions about ab-test-setup

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I calculate sample size and duration for an A/B test?▼

A valid A/B testing hypothesis predicts a specific outcome based on a proposed change. You must define a clear hypothesis, your primary conversion metric, and guardrail metrics to measure the actual impact of your variants accurately.

Can I run multivariate tests for conversion optimization alongside pricing changes?▼

You need your baseline conversion rate, minimum detectable effect, primary metric, and guardrail metrics. Providing these inputs allows the experiment design to calculate the necessary sample size and traffic split for statistically valid results.

What's the best way to analyze split testing results and decide whether to ship?▼

The best way to analyze split testing results is to check statistical significance, review segment behavior, and evaluate guardrail metrics. This process determines whether the conversion improvement is real or just random noise before you ship.

Why does my A/B test need guardrail metrics in addition to a primary metric?▼

A/B testing needs guardrail metrics alongside a primary metric to prevent unintended negative impacts on your marketing workflows. While the primary metric measures the target improvement, guardrails ensure overall business health remains stable.

Related Skills

Explore More Marketing & Sales Skills→
tapanshahtapanshah

campaign-analytics

Analyze marketing campaign performance with multi-touch attribution, funnel rates, and ROI calculations.

Community
Advanced
w95w95
145

statistical-analysis

Apply descriptive statistics, trend analysis, outlier detection, and hypothesis testing to datasets.

Community
Intermediate
VictaaawVictaaaw

ab-test-setup

Plan and execute statistically valid A/B and growth experiments.

Community
Advanced
giosuetedeschi-specgiosuetedeschi-spec

ab-test-setup

Design and execute statistically valid A/B, A/B/n, and multivariate tests.

Community
Intermediate