skills.rest
apexradius
apexradiusOFFICIAL
Explore all skills by this author →
WebsiteGitHub

ab-test

Calculate sample size, design, and analyze A/B tests for statistical significance.

Updated Jun 21, 2026
Search Tags:#data science#test design#a/b testing#statistical analysis#sample size
One-click install
npx skills add https://github.com/apexradius/agentic-os --skill ab-test-apexradius
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ab-test
Source: https://github.com/apexradius/agentic-os/tree/main/skills/ab-test
Command: npx skills add https://github.com/apexradius/agentic-os --skill ab-test-apexradius

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill simplifies the planning and analysis of A/B tests, ensuring you have the necessary sample size and statistical significance to make informed decisions.

Core Features & Use Cases

  • Sample Size Calculator: Estimate the required sample size to detect a meaningful effect.
  • Test Design: Outline the structure of your A/B test, including variants and metrics.
  • Statistical Significance Analysis: Evaluate the results of your A/B test for statistical significance.
  • Test Prioritization: Score tests based on impact, confidence, and ease to prioritize test execution.
  • Common Test Types: Guidance on different types of A/B tests and their impact levels.
  • Anti-Patterns: Avoid common pitfalls in A/B testing to ensure reliable results.
  • Output: Provide a structured summary of the test results with recommendations.
  • Use Case: Imagine you are running an A/B test on a website's checkout button color. Use this Skill to determine the required sample size, analyze the test results, and decide whether to keep the new button color or revert to the original.

Quick Start

Analyze the results of your A/B test with the ab-test skill.

Frequently Asked Questions about ab-test

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I calculate sample size for an A/B test?▼

To calculate sample size for an A/B test, you estimate the required number of participants needed to detect a meaningful effect. This Skill provides a sample size calculator to ensure your test has sufficient statistical power for reliable results.

How do I analyze A/B test results for statistical significance?▼

Analyzing A/B test results for statistical significance involves evaluating your variant data against the original to determine if observed differences are meaningful. This Skill processes your test data to provide a structured summary with recommendations.

What is the best way to prioritize A/B tests for product development?▼

Prioritizing A/B tests involves scoring test ideas based on impact, confidence, and ease. This Skill provides test prioritization capabilities to help you sequence test execution for optimal product development outcomes.

Do I need statistical knowledge to design A/B tests?▼

Yes, designing A/B tests requires statistical knowledge and an understanding of A/B testing principles. This Skill is intended for use in marketing, product development, and UX testing but requires prerequisite knowledge to interpret results accurately.

What are common anti-patterns to avoid in A/B testing?▼

Common anti-patterns in A/B testing include pitfalls that compromise reliable results, such as insufficient sample sizes or misinterpreted metrics. This Skill offers guidance to avoid these errors and ensure your statistical analysis remains valid.

Related Skills

Explore More Data & Analytics Skills→
loujcloujc

analyze-results

Parse JSON/CSV experiment results and compute metrics versus baseline.

Community
Advanced
juandaniel190juandaniel190
1

statistical-analysis

Apply descriptive statistics, trend analysis, outlier detection, and hypothesis testing to datasets using Python libraries for calculations and visualizations.

Community
Intermediate
stratagastrataga

ab-test-setup

Design statistically valid A/B tests with sample-size calculations and result interpretation.

Community
Advanced
assimovtassimovt
54

experiment-design

Design hypothesis-driven A/B tests with statistical rigor and pre-committed plans.

Community
Intermediate