skills.rest
strataga
stratagaCOMMUNITY
Explore all skills by this author →
Website𝕏@jcochranioGitHub

ab-test-setup

Design statistically valid A/B tests with sample-size calculations and result interpretation.

Updated Jan 23, 2026
Search Tags:#A/B testing#metrics#hypothesis#ab-testing#sample-size#experimental-design#statistical-significance
One-click install
npx skills add https://github.com/strataga/claude-skill-inception --skill ab-test-setup-strataga
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ab-test-setup
Source: https://github.com/strataga/claude-skill-inception/tree/main/ab-test-setup
Command: npx skills add https://github.com/strataga/claude-skill-inception --skill ab-test-setup-strataga

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill helps teams design and run statistically sound A/B tests to determine the impact of changes on product performance.

Core Features & Use Cases

  • Hypothesis framing: Create clear, testable hypotheses with defined success metrics.
  • Experiment planning: Guidance on sample size, duration, traffic allocation, and milestones.
  • Analysis & learning: Structured documentation of results, takeaways, and recommended next steps.
  • Use Case: Plan an A/B test for a homepage element to measure lift in conversions.

Quick Start

Design an A/B test for the homepage signup CTA with a 5% MDE, 95% confidence, and generate a test plan document.

Frequently Asked Questions about ab-test-setup

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I calculate sample size for an A/B test with a 5% minimum detectable effect?▼

To calculate sample size for an A/B test, you need to define your minimum detectable effect (MDE), statistical confidence level, and baseline conversion metrics. This Skill uses those inputs to generate a structured experiment plan with appropriate traffic allocation and test duration.

What is the best way to formulate a testable hypothesis for an A/B test?▼

A testable A/B testing hypothesis requires a clear statement of the expected change, the specific element being modified, and the defined success metrics used to measure impact. This Skill provides structured guidance to frame hypotheses that directly align with your selected metrics.

Does this A/B testing guidance support multivariate tests (MVT) and A/B/n experiments?▼

Yes, this Skill supports planning multiple test types including standard A/B tests, A/B/n experiments with multiple variants, and multivariate tests (MVT). It helps you determine the appropriate experimental design based on your specific metrics and hypothesis requirements.

How do I interpret statistical significance and results after running an A/B test?▼

Interpreting A/B testing results involves analyzing statistical significance against your predefined confidence levels and success metrics. This Skill provides structured documentation of outcomes, takeaways, and recommended next steps to ensure valid conclusions from your experiment data.

Related Skills

Explore More Data & Analytics Skills→
Peter-Swain-IncPeter-Swain-Inc
1

conversion-rate-optimiser

Analyze funnel drop-offs and configure conversion optimization workflows.

Official
Advanced
spandiosspandios

ab-test-analysis

Analyze A/B test results for statistical significance and actionable recommendations.

Community
Advanced
traygerbigtraygerbig
1

Agent Dashboard

Monitor OpenClaw agent performance and analytics in real-time.

Community
Advanced
systemifyautomationsystemifyautomation

ab-test-setup

Plan and design A/B tests with hypothesis generation and sample size calculation.

Community
Intermediate