One-click install
npx skills add https://github.com/andginja/marketingskills --skill ab-test-setup-andginja
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: ab-test-setup
Source: https://github.com/andginja/marketingskills/tree/main/skills/ab-test-setup
Command: npx skills add https://github.com/andginja/marketingskills --skill ab-test-setup-andginja

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill helps you design, execute, and analyze A/B tests with statistical rigor, ensuring your experiments yield reliable insights and avoid common pitfalls.

Core Features & Use Cases

  • Hypothesis Formulation: Guides you in creating clear, testable hypotheses using a structured template.
  • Prioritization Frameworks: Implements ICE, PIE, and RICE frameworks to rank test ideas by potential impact and feasibility.
  • Sample Size & Duration Calculation: Provides tools and tables to determine the necessary sample size and test duration for statistical validity.
  • Mistake Avoidance: Educates on common A/B testing errors like peeking, multiple comparisons, and underpowered tests, offering solutions.
  • Use Case: You have a new landing page design and want to know if it will increase sign-ups. This Skill will help you formulate a hypothesis, calculate how many visitors you need to test, and determine how long to run the test to get a statistically significant result.

Quick Start

Help me design an A/B test to increase sign-ups on our landing page by formulating a hypothesis and calculating the required sample size.

Frequently Asked Questions about ab-test-setup

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I calculate sample size and duration for an A/B test?

Calculating A/B test sample size and duration requires determining the necessary visitor count and test runtime to achieve statistical validity and avoid underpowered tests. This Skill provides tables and tools to compute these metrics for reliable conversion rate optimization.

What are common A/B testing mistakes like peeking and multiple comparisons?

Common A/B testing mistakes include peeking at results early, running multiple comparisons without adjustment, and using underpowered tests. This Skill educates on these statistical errors and offers solutions to ensure your experimentation yields reliable insights.

How do I formulate a strong hypothesis for conversion rate optimization?

Formulating a strong A/B testing hypothesis involves using a structured template to create clear, testable predictions for growth marketing. This Skill guides you through hypothesis formation to ensure your conversion rate optimization experiments are statistically sound.

Which framework should I use to prioritize A/B test ideas?

Prioritizing A/B test ideas can be done using ICE, PIE, or RICE frameworks to rank experiments by potential impact, confidence, and feasibility. This Skill implements these prioritization frameworks to help you sequence your growth marketing tests effectively.

Does this Skill support both frequentist and Bayesian approaches to A/B testing?

Yes, this Skill supports interpreting A/B test results using both frequentist and Bayesian statistical approaches. It guides you through hypothesis testing and experimentation analysis using whichever method suits your data.

When should I not run an A/B test?

You should avoid A/B testing when you lack the traffic to reach the required sample size, leading to underpowered tests and inconclusive statistics. This Skill helps identify these edge cases to prevent wasted experimentation effort.