skills.rest
Gull-Stack
Gull-StackOFFICIAL
Explore all skills by this author →
WebsiteGitHub

ab-test-setup

Designs, plans, and analyzes A/B tests including hypothesis, metrics, and sample size calculations.

Updated Jan 31, 2026
Search Tags:#planning#metrics#statistics#hypothesis#ab-testing#experimental-design
One-click install
npx skills add https://github.com/Gull-Stack/love-rescue --skill ab-test-setup-gull-stack
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ab-test-setup
Source: https://github.com/Gull-Stack/love-rescue/tree/main/.claude/skills/ab-test-setup
Command: npx skills add https://github.com/Gull-Stack/love-rescue --skill ab-test-setup-gull-stack

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill helps product teams and analysts structure and run rigorous A/B tests to determine which changes drive measurable improvements, reducing guesswork and random variation.

Core Features & Use Cases

  • Hypothesis framing: Create a clear, testable statement linking a change to an expected outcome.
  • Experiment planning: Define primary and secondary metrics, sample size, duration, and traffic allocation.
  • Variant design: Outline controls and variants with clear descriptions and success criteria.
  • Documentation & learning: Capture results, learnings, and next steps for institutional knowledge.
  • Use Case: Launching a landing-page variant to lift signup rate by a predefined MDE while monitoring guardrail metrics.

Quick Start

Draft a basic A/B test plan by defining the hypothesis, selecting a primary metric, calculating the sample size, and outlining control and variant descriptions.

Frequently Asked Questions about ab-test-setup

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I calculate sample size and test duration for an A/B test?▼

A/B test sample size and duration planning require defining your primary metric, minimum detectable effect, and traffic allocation to reach statistical significance. This skill calculates these parameters while generating a complete experimental timeline.

How do I frame a hypothesis for an A/B test experiment?▼

A/B test hypothesis framing creates a clear, testable statement linking a specific product change to an expected measurable outcome. This skill structures your hypothesis to define success criteria and guide variant design.

Can I plan multipath variants and guardrail metrics for mobile and web experiments?▼

Multipath variants and guardrail metrics are supported for web and mobile product changes, marketing copy, and pricing experiments. This skill outlines multiple test variants while monitoring secondary metrics to prevent negative impacts.

What is the best way to document A/B test results and learnings?▼

Documenting A/B test results captures outcomes, institutional knowledge, and next steps for future experiments. This skill provides a structured format to record learnings from your control and variant performances.

Does this A/B testing skill support both client-side and server-side execution planning?▼

Client-side and server-side execution planning are both supported within the implementation plan generated for your experiment. This skill defines the technical execution approach based on your specific rollout requirements.

Related Skills

Explore More Data & Analytics Skills→
shalevaminshalevamin

metrics-dashboard

Define product metrics dashboards with North Star, Input, Health, and Business layers.

Community
Intermediate
runchr-comrunchr-com

ab-test-analysis

Analyze A/B test results for statistical significance and decision recommendations.

Official
Intermediate
omeraltnomeraltn

agency-livestream-commerce-coach

Optimize livestream e-commerce host operations and traffic strategy for Douyin, Kuaishou, Taobao Live, and Channels.

Community
Advanced
dxprdxpr
1

RL: Reinforcement Learning Experiments (Drush CLI)

Manage Thompson Sampling multi-armed bandit experiments via Drush commands.

Official
Advanced