ab-test-setup

Plan and run statistically valid A/B, multivariate, and split URL experiments.

Updated Apr 6, 2026
One-click install
npx skills add https://github.com/sneha835/Sneha --skill ab-test-setup-sneha835
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: ab-test-setup
Source: https://github.com/sneha835/Sneha/tree/main/.claude/skills/ab-test-setup
Command: npx skills add https://github.com/sneha835/Sneha --skill ab-test-setup-sneha835

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This skill helps product, marketing, and growth teams design and run statistically valid experiments to determine the best performing changes, reducing guesswork and accelerating evidence-based decisions.

Core Features & Use Cases

  • Hypothesis-driven test design using a formal structure (Because observations/data, we believe changes will cause outcomes).
  • Support for test types including A/B, A/B/n, MVT, and split URL to match various experimentation needs.
  • Guidance on sample size, duration, power, and how to define primary, secondary, and guardrail metrics.
  • Guidance on variant design, traffic allocation, implementation approaches (client-side vs server-side), and result interpretation.
  • Templates and tooling for documentation, playbooks, and ongoing growth experimentation programs to build reusable patterns.

Quick Start

Define the test context and baseline metrics, write a clear hypothesis, choose a suitable test type, calculate the required sample size, implement the variants, and start the experiment with predefined success criteria.

Frequently Asked Questions about ab-test-setup

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I calculate sample size and duration for an A/B test?

To calculate A/B test sample size and duration, you need baseline conversion rates, minimum detectable effect, and statistical power. This skill enforces formal pre-commit duration planning and sample-size calculation to ensure tests reach statistical significance.

What is the best way to structure a hypothesis for conversion rate experimentation?

The best way to structure a conversion rate experimentation hypothesis is using a formal structure: "Because observations/data, we believe changes will cause outcomes." This skill enforces this hypothesis-driven test design to reduce guesswork in growth experiments.

Can I use this skill for multivariate testing and split URL experiments?

Yes, you can use this skill for multivariate testing and split URL experiments. It supports A/B, A/B/n, MVT, and split URL testing across product, marketing, and growth contexts to help determine the best performing changes.

How do I define primary, secondary, and guardrail metrics for an experiment?

Defining primary, secondary, and guardrail metrics requires aligning them with your experiment's goals and risk thresholds. This skill provides guidance on selecting these metrics to ensure rigorous result interpretation and protect against negative downstream impacts.

When should I choose multivariate testing over a standard A/B test?

You should choose multivariate testing over a standard A/B test when you need to understand the interaction effects between multiple changes simultaneously. This skill helps determine the suitable test type based on your specific experimentation needs and variant design.