skills.rest
batdorj-s
batdorj-sCOMMUNITY
Explore all skills by this author →
GitHub

ab-testing

Design, implement, and analyze A/B tests with statistical guidance.

Updated Jul 5, 2026
Search Tags:#conversion optimization#experimentation#a/b testing#ab testing#growth experimentation
One-click install
npx skills add https://github.com/batdorj-s/tlj --skill ab-testing-batdorj-s
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ab-testing
Source: https://github.com/batdorj-s/tlj/tree/main/.agents/skills/ab-testing
Command: npx skills add https://github.com/batdorj-s/tlj --skill ab-testing-batdorj-s

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill provides comprehensive guidance for planning, designing, and implementing A/B tests and growth experimentation programs, helping users measure the performance of two approaches and build systematic experimentation practices.

Core Features & Use Cases

  • A/B Test Setup: Offers a structured approach to setting up A/B tests, including initial assessment, core principles, and hypothesis framework.
  • Test Types: Explains different types of tests (A/B, A/B/n, MVT, Split URL) and their respective traffic requirements.
  • Sample Size: Provides a quick reference table for sample size calculation based on baseline conversion rate and desired lift.
  • Metrics Selection: Guides on selecting primary, secondary, and guardrail metrics for effective test analysis.
  • Growth Experimentation Program: Offers insights into building a continuous experimentation program with an experiment loop, hypothesis generation, and prioritization.
  • Common Mistakes: Lists common mistakes in test design, execution, and analysis to avoid.
  • Task-Specific Questions: Answers specific questions related to A/B testing, such as sample size calculation, metric selection, and test interpretation.

Quick Start

Use the ab-testing skill to get expert advice on designing an A/B test for your website.

Frequently Asked Questions about ab-testing

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I calculate sample size for an A/B test based on baseline conversion rate?▼

Calculate A/B test sample size by referencing a quick lookup table using your baseline conversion rate and desired lift to determine the required traffic volume per variation.

What is the difference between A/B/n, MVT, and Split URL testing?▼

A/B/n tests multiple variants, MVT evaluates multiple element combinations simultaneously, and Split URL redirects traffic to different pages, each requiring distinct traffic volumes.

How do I select primary, secondary, and guardrail metrics for growth experimentation?▼

Select primary metrics to measure the main test goal, secondary metrics for indirect effects, and guardrail metrics to prevent negative impacts on critical business indicators during experimentation.

What are the most common mistakes in A/B test design and analysis?▼

Common A/B testing mistakes include stopping tests too early before reaching significance, testing too many variations without enough traffic, and ignoring novelty effects on results.

How do I build a continuous growth experimentation program?▼

Build a continuous growth experimentation program by establishing an experiment loop that systematically generates, prioritizes, and tests hypotheses to drive ongoing conversion optimization.

What statistical concepts do I need to know for conversion optimization testing?▼

Conversion optimization testing requires understanding statistical concepts like significance, confidence intervals, and variance to accurately interpret A/B test results and avoid false conclusions.

Related Skills

Explore More Product & Management Skills→
ryzamediaryzamedia
1

ab-testing

Design and evaluate statistically rigorous A/B and multivariate experiments.

Community
Advanced
lilbom32lilbom32

statistical-analysis

Analyze data distributions, variability, and statistical significance for decision-making.

Community
Advanced
snoodleboot-iosnoodleboot-io
2

grafana-dashboard-design-minimal

Design minimal Grafana dashboard layouts with predefined sections and visualization types.

Official
Basic
kiryteokiryteo
1

user-research-synthesis

Synthesize user research into themes, insights, and recommended actions.

Community
Advanced