skills.rest
cipriantitire
cipriantitireCOMMUNITY
Explore all skills by this author →
Website𝕏@psyccotoxiccGitHub

ab-test-setup

Design statistically valid A/B tests with sample size and duration calculations.

Updated Aug 27, 2026
Search Tags:#metrics#statistics#hypothesis#test-design#ab-testing#sample-size#a/b-testing
One-click install
npx skills add https://github.com/cipriantitire/noctvm --skill ab-test-setup-cipriantitire
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ab-test-setup
Source: https://github.com/cipriantitire/noctvm/tree/main/Skills/marketingskills-main/skills/ab-test-setup
Command: npx skills add https://github.com/cipriantitire/noctvm --skill ab-test-setup-cipriantitire

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

A/B test setup helps teams structure experiments to determine which changes reliably improve key metrics, avoiding guesswork and vanity metrics.

Core Features & Use Cases

  • Hypothesis-driven test design and planning
  • Sample size and duration calculation with guidance from references
  • Primary/secondary/guardrail metric definitions and interpretation
  • Variant design, traffic allocation, and test types (A/B, A/B/n, MVT)
  • Documentation templates for planning, results, and handoffs

Quick Start

Draft a complete A/B test plan for your page, including hypothesis, variants, metrics, sample size, and duration.

Frequently Asked Questions about ab-test-setup

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I calculate the sample size for an A/B test?▼

A/B test design requires a clear hypothesis, defined variants, and categorized metrics to determine winning changes. This skill enforces structured planning by guiding you through hypothesis creation, variant design, and traffic allocation for marketing and CRO contexts.

What is the best way to structure A/B test hypotheses and metrics?▼

A/B test design requires a clear hypothesis, defined variants, and categorized metrics to determine winning changes. This skill enforces structured planning by guiding you through hypothesis creation, variant design, and traffic allocation for marketing and CRO contexts.

How do I interpret A/B test results and statistical significance?▼

Interpreting A/B test results involves evaluating primary, secondary, and guardrail metrics against your calculated significance thresholds. This skill provides evaluation templates to help you analyze variant performance and determine if a change reliably improves key metrics.

Can I use this for multivariate testing and A/B/n test designs?▼

Yes, you can use this for A/B, A/B/n, and multivariate testing (MVT) designs. The skill supports variant design and traffic allocation across multiple test types to help you compare page variations in product optimization scenarios.

Why do I need guardrail metrics for my A/B testing plan?▼

Guardrail metrics are needed in your A/B testing plan to prevent unintended negative impacts on key business metrics while optimizing primary targets. This skill helps you define and monitor these metrics alongside primary and secondary ones to avoid vanity results.

Related Skills

Explore More Marketing & Sales Skills→
X-McKayX-McKay
3

detect-metric-anomaly

Detect statistical anomalies in time-series metrics using z-score calculations and thresholds.

Community
Intermediate
hiroshi75hiroshi75

statistical-analysis

Apply descriptive statistics, trend analysis, outlier detection, and hypothesis testing to datasets.

Community
Intermediate
gerald-icagerald-ica

ab-test-setup

Plan, run, and analyze A/B tests with statistical significance.

Community
Advanced
stratagastrataga

ab-test-setup

Design statistically valid A/B tests with sample-size calculations and result interpretation.

Community
Advanced