agency-experiment-tracker

Sample size calculation for A/B testing requires defining your hypothesis, target confidence level, and statistical power to ensure reliable results. This Skill computes required sample sizes and applies power calculations to guarantee your experiment can detect meaningful effects.

Multiple comparison correction controls false positives when running multi-variate experiments or testing several metrics simultaneously. This Skill applies statistical corrections during analysis to maintain the integrity of your hypothesis testing and prevent inflated significance rates.

Feature rollout monitoring requires defining instrumentation checklists, guardrail metrics, and clear rollback triggers before launch. This Skill structures your execution tracking by specifying monitoring criteria and safety procedures to protect users during the experiment.

Multi-variate experiments are fully supported alongside standard A/B testing workflows. This Skill structures complex experiment designs by formulating testable hypotheses, selecting appropriate statistical tests, and applying multiple comparison corrections across all variations.

An experiment decision report should combine confidence intervals, effect sizes, and segment analyses to justify the final recommendation. This Skill produces post-test analysis and clear go/no-go criteria so product teams can make defensible, data-driven rollout decisions.