skills.rest
Anwarito-lbd
Anwarito-lbdCOMMUNITY
Explore all skills by this author →
GitHub

ab-test-setup

Plan and run statistically valid A/B tests with sample size calculations.

Updated Nov 12, 2025
Search Tags:#metrics#hypothesis#experimentation#ab-testing#sample-size#variant-design
One-click install
npx skills add https://github.com/Anwarito-lbd/conversioncraft --skill ab-test-setup-anwarito-lbd
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ab-test-setup
Source: https://github.com/Anwarito-lbd/conversioncraft/tree/main/.marketingskills/skills/ab-test-setup
Command: npx skills add https://github.com/Anwarito-lbd/conversioncraft --skill ab-test-setup-anwarito-lbd

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This skill helps teams plan and run statistically valid A/B tests and experiments to derive actionable insights that drive product and marketing decisions.

Core Features & Use Cases

  • Provide a structured path from hypothesis to decision, including test type selection (A/B, A/B/n, MVT, split URL), sample size estimation, metric selection, and variant design.
  • Support running tests with clear implementation guidance (client-side or server-side) and running a comprehensive post-test analysis and documentation workflow.
  • Use cases include optimizing conversion rates, comparing pricing or copy, testing UI layouts, and validating product changes before wide rollout.

Quick Start

Plan and document a complete A/B test plan including hypothesis, variant ideas, sample size, runtime, metrics, and decision criteria.

Frequently Asked Questions about ab-test-setup

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I calculate sample size for an A/B test to get statistically valid results?▼

A/B test sample size calculation requires defining your hypothesis, selecting metrics, and estimating baseline conversion rates to ensure statistically valid results and derive actionable product or marketing insights.

What's the best way to structure an A/B test hypothesis for product optimization?▼

Structure an A/B test hypothesis by following a structured path from hypothesis to decision, defining expected changes, target metrics, and variant design to effectively optimize conversion rates and validate product changes.

When do I need multivariate testing instead of a standard A/B/n test?▼

Use multivariate testing (MVT) instead of an A/B/n test when you need to test multiple variables and their interactions simultaneously, whereas A/B/n compares multiple distinct variants against a control without evaluating interaction effects.

Can I use this approach for both client-side and server-side experimentation?▼

Yes, you can use this approach for both client-side and server-side experimentation, providing clear implementation guidance for accurate variant delivery when testing UI layouts, pricing, or copy changes.

How do I interpret A/B test results and document post-test analysis?▼

Interpret A/B test results by running a comprehensive post-test analysis and documentation workflow, evaluating metric performance against decision criteria to translate experiment data into actionable insights for rollout decisions.

Related Skills

Explore More Data & Analytics Skills→
raphaelmansraphaelmans

ab-test-setup

Generate statistically valid A/B, A/B/n, MVT, and split URL test plans with sample-size calculations.

Community
Advanced
SpaceZephyrSpaceZephyr
14

SPACE-analytics

Transform raw product data into interactive HTML analysis reports.

Community
Advanced
homostellarishomostellaris
2

nextjs-app-router

Integrate PostHog analytics into Next.js App Router applications.

Community
Intermediate
lebongobenilebongobeni

ab-test-setup

Design statistically valid A/B tests and experiment plans for marketing pages.

Community
Advanced