skills.rest
soberzerg
soberzergCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

ab-test-setup

Plan, design, and implement A/B tests with hypothesis formulation and metric selection.

Updated Jan 2, 2026
Search Tags:#hypothesis testing#experimentation#a/b testing#product analytics#conversion rate optimization#split testing
One-click install
npx skills add https://github.com/soberzerg/second-brain --skill ab-test-setup-soberzerg
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ab-test-setup
Source: https://github.com/soberzerg/second-brain/tree/main/.agents/skills/ab-test-setup
Command: npx skills add https://github.com/soberzerg/second-brain --skill ab-test-setup-soberzerg

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill helps users plan, design, and implement A/B tests or experiments to optimize product features, marketing copy, or user interfaces, ensuring statistically valid and actionable results.

Core Features & Use Cases

  • Hypothesis Formulation: Guides users to create strong, testable hypotheses using a clear framework.
  • Test Design: Provides principles for choosing test types, defining metrics (primary, secondary, guardrail), and designing variants.
  • Sample Size & Duration Calculation: Offers guidance and references for determining necessary sample sizes and estimating test duration.
  • Implementation & Analysis: Outlines best practices for running tests and analyzing results for statistical significance and practical impact.
  • Use Case: A product manager wants to test a new headline on the homepage to increase sign-ups. This Skill will help them define a hypothesis, choose metrics, calculate the required sample size, and understand how to analyze the results.

Quick Start

Use the ab-test-setup skill to help plan an A/B test for a new call-to-action button on the pricing page.

Frequently Asked Questions about ab-test-setup

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I design an A/B test for product optimization?▼

To design an A/B test, you must formulate a testable hypothesis, select primary and guardrail metrics, calculate the required sample size, and define variant designs for statistically valid product optimization results.

What is the best way to calculate sample size and duration for split testing?▼

Calculating sample size and duration for split testing requires selecting your primary metrics and using statistical references to ensure your experiment reaches sufficient practical significance and statistical rigor before analysis.

When should I use multivariate testing instead of A/B/n testing?▼

You should use multivariate testing (MVT) instead of A/B/n testing when you need to evaluate multiple variables simultaneously, whereas A/B/n testing compares multiple variants against a single baseline.

How do I formulate a strong hypothesis for conversion rate optimization experiments?▼

Formulating a strong hypothesis for conversion rate optimization involves using a clear framework to define the expected change, the targeted metric, and the underlying reasoning before running the experiment.

What metrics do I need to track for an effective conversion rate optimization experiment?▼

For effective conversion rate optimization, you need to track primary metrics to measure success, secondary metrics for context, and guardrail metrics to prevent negative impacts on other product areas.

Related Skills

Explore More Product & Management Skills→
sQVesQVe

debug-issue

Diagnose bugs by tracing evidence and generating a BR issue template.

Community
Advanced
product-on-purposeproduct-on-purpose
527

measure-experiment-results

Document A/B test results with metrics, learnings, and recommendations.

Official
Advanced
LuminaVaultLuminaVault

market-alert-agent

Monitor financial markets and send configurable alerts via Discord, Telegram, or Slack.

Official
Advanced
Veloxia-agencyVeloxia-agency

ab-test-setup

Design statistically valid A/B tests with hypotheses, metrics, and sample sizes.

Community
Advanced