ab-testing

Design and analyze A/B tests for product development and feature rollout.

Updated Aug 27, 2026
One-click install
npx skills add https://github.com/coco-de/skills --skill ab-testing-coco-de
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: ab-testing
Source: https://github.com/coco-de/skills/tree/main/plugins/cc-pm-analytics/skills/ab-testing
Command: npx skills add https://github.com/coco-de/skills --skill ab-testing-coco-de

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill helps you design, execute, and analyze A/B tests to make data-driven decisions about feature and UI changes, ensuring you understand their impact.

Core Features & Use Cases

  • Experiment Design: Formulate hypotheses, define primary and guardrail metrics, and plan for user segmentation.
  • Sample Size Calculation: Determine the necessary sample size and experiment duration based on statistical power and significance.
  • Analysis & Decision Making: Validate statistical significance, calculate effect sizes, and make informed "Ship/Don't Ship" decisions.
  • Use Case: You've developed two different onboarding flows for new users. Use this Skill to design an A/B test to determine which flow leads to higher user retention.

Quick Start

Design an A/B test for a new checkout button color, hypothesizing it will increase conversion rate by 5%.

Frequently Asked Questions about ab-testing

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I calculate sample size and duration for an A/B test?

To calculate A/B test sample size and duration, you need to define your primary metrics, then determine the necessary sample size based on target statistical power and significance levels to ensure valid experiment execution.

What is the best way to design an A/B test for a new feature rollout?

Designing an A/B test for feature rollout requires formulating a clear hypothesis, defining primary and guardrail metrics for user segmentation, and planning event logging integration to capture the impact of UI changes accurately.

How do I interpret A/B test results for data-driven product decisions?

Interpreting A/B test results involves validating statistical significance, calculating effect sizes from your experiment data, and using these metrics to make informed ship or don't ship decisions for product development.

Does A/B testing work with feature flagging systems on mobile and web platforms?

Yes, A/B testing supports mobile and web platforms by integrating with feature flagging and event logging systems, allowing you to execute experiments and analyze user behavior across different environments.

When do I need to use guardrail metrics in experimentation?

You need guardrail metrics during experimentation to monitor and prevent negative impacts on existing user behavior while testing new features, ensuring your primary metric improvements do not harm overall product health.