experiment-designer

Plan, execute, and interpret product experiments with sample-size calculations.

Updated Apr 16, 2026

Search Tags:#a/b testing #hypotheses #sample-size #experiment-design #ice-scoring #product-decisions

Ready to run

npx skills add https://github.com/devCharuzu/philfida-taskmanage --skill experiment-designer-devcharuzu

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: experiment-designer
Source: https://github.com/devCharuzu/philfida-taskmanage/tree/main/.windsurf/skills/experiment-designer
Command: npx skills add https://github.com/devCharuzu/philfida-taskmanage --skill experiment-designer-devcharuzu

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

Planning and running product experiments with clearly defined hypotheses, defensible sample-size estimates, and structured interpretation to inform decisions.

Core Features & Use Cases

Write testable hypotheses using If/Then/Because format to guide experimentation.
Estimate sample size, define MDE, set alpha/power, and prioritize tests with ICE scoring.
Interpret results with surveillance of p-values, confidence intervals, and practical significance; reference playbooks for best practices.

Quick Start

Propose a hypothesis for a new feature and run the sample-size calculator to plan the test.

Frequently Asked Questions about experiment-designer

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I calculate sample size for a/b testing?▼

Sample size for a/b testing is calculated by defining the minimum detectable effect, alpha, and power, then running the Skill's Python tooling script to generate defensible test estimates.

What is the best way to write a testable hypothesis for product experiments?▼

Testable hypotheses for product experiments use an If/Then/Because format to clearly link the proposed change to the expected outcome, ensuring structured and rigorous experimentation design.

Can I use ICE scoring to prioritize product experiments?▼

Yes, ICE scoring is integrated into the experiment design workflow to prioritize product experiments, helping sequence a/b and multivariate tests based on impact, confidence, and ease.

How do I interpret a/b test results with confidence intervals and p-values?▼

Interpreting a/b test results involves surveilling p-values and confidence intervals to determine practical significance, guided by built-in references to experiment best-practices and stopping rules.

Does this support multivariate and holdout tests, or just a/b testing?▼

It supports multivariate and holdout tests alongside a/b testing, allowing you to plan, execute, and interpret product experiments across web, mobile, and feature changes.