Experiment

Design, analyze, and report on A/B tests with statistical significance methods.

68|14|Updated Jan 7, 2026
One-click install
npx skills add https://github.com/simota/agent-skills --skill experiment-simota
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: Experiment
Source: https://github.com/simota/agent-skills/tree/main/experiment
Command: npx skills add https://github.com/simota/agent-skills --skill experiment-simota

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill helps you rigorously test product hypotheses and make data-backed decisions by designing, executing, and analyzing experiments.

Core Features & Use Cases

  • A/B Test Design: Create statistically sound A/B tests to compare different product variations.
  • Hypothesis Validation: Formulate and test falsifiable hypotheses with clear success criteria.
  • Sample Size Calculation: Determine the necessary sample size for reliable results.
  • Statistical Analysis: Analyze experiment results to determine statistical significance.
  • Use Case: You have a new checkout flow design. Use this Skill to design an A/B test, calculate the required sample size, analyze the results, and determine if the new flow leads to a statistically significant increase in conversion rates.

Quick Start

Design an A/B test for a new signup button color, with a hypothesis that it will increase signups by 5%.

Frequently Asked Questions about Experiment

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I calculate sample size for A/B testing a new checkout flow?

Sample size calculation for A/B testing requires defining your baseline conversion rate, minimum detectable effect, and statistical power. This Skill computes the necessary sample size to ensure reliable results before experiment execution.

What is the best way to analyze statistical significance in product analytics experiments?

Statistical significance in product analytics experiments is analyzed using methods like Z-tests and sequential testing. This Skill evaluates your experiment data to determine if observed metric changes are statistically valid rather than random variance.

Can I use feature flagging to manage multivariate testing variations?

Feature flagging facilitates multivariate testing by managing multiple experiment variations simultaneously. This Skill supports feature flag implementation to control exposure and route traffic to different product variations during testing.

How do I formulate a falsifiable hypothesis with clear success criteria for experimentation?

Hypothesis formulation for experimentation requires defining a falsifiable prediction with clear success criteria. This Skill facilitates hypothesis document creation to ensure your product tests are structured, measurable, and statistically rigorous.

Does statistical analysis for A/B testing work without external dependencies?

Statistical analysis for A/B testing operates with no external dependencies. This Skill handles significance testing, sample size calculation, and result reporting internally to provide actionable recommendations based on your experiment data.