hypothesis-testing

Design and analyze hypothesis tests for experiments and A/B tests.

Updated Apr 8, 2026
One-click install
npx skills add https://github.com/zm2231/personal-os-cowork --skill hypothesis-testing-zm2231
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: hypothesis-testing
Source: https://github.com/zm2231/personal-os-cowork/tree/main/.claude/skills/hypothesis-testing
Command: npx skills add https://github.com/zm2231/personal-os-cowork --skill hypothesis-testing-zm2231

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Helps researchers and product teams design valid experiments, choose appropriate statistical tests, calculate sample size and power, and interpret results so decisions are based on sound evidence rather than misapplied statistics or p-hacking.

Core Features & Use Cases

  • Hypothesis formulation & test selection: Define null and alternative hypotheses, choose one-tailed vs two-tailed, and recommend parametric or non-parametric tests (t-test, chi-square, ANOVA, z-test for proportions).
  • Power analysis & sample size: Perform a priori and post-hoc power calculations to determine required sample sizes and trade-offs between detectable effect and cost.
  • Execution & interpretation: Calculate p-values, confidence intervals, effect sizes, provide APA-style reporting, and offer Bayesian and causal inference options for advanced analyses.
  • Use Case: Plan and analyze an A/B test for a website redesign, compute required visitors per variant, run the selected statistical test on collected counts, and produce an actionable recommendation.

Quick Start

Design a one-tailed A/B test to detect a 20% relative uplift from a 3.2% baseline with 80% power and return the required sample size, test choice, and analysis plan.

Frequently Asked Questions about hypothesis-testing

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I calculate sample size and power for an A/B test?

Sample size and power analysis for an A/B test requires performing a priori calculations to determine required visitors per variant, balancing detectable effect sizes against testing costs to ensure 80% power.

What statistical test should I use for my experimental data?

Choosing the right statistical test depends on your data type and distribution, allowing you to select parametric or non-parametric options like t-tests, chi-square, ANOVA, or z-tests for proportions.

Can I use Bayesian inference instead of traditional p-value hypothesis testing?

Yes, Bayesian inference can be used instead of traditional p-value hypothesis testing to analyze experimental results, offering an advanced alternative for interpreting evidence and calculating effect sizes.

How do I interpret A/B test results and avoid p-hacking?

Interpreting A/B test results involves calculating p-values, confidence intervals, and effect sizes to produce APA-style reports, ensuring product decisions are based on sound evidence rather than misapplied statistics.

Does this tool support causal inference methods for experimental analysis?

Yes, causal inference methods are supported for advanced experimental analyses, allowing researchers to move beyond basic hypothesis testing and determine cause-and-effect relationships from collected data.

Why do I need summary statistics or raw event counts to run a hypothesis test?

Summary statistics or raw event counts are required as inputs to run a hypothesis test because they provide the underlying data necessary to execute calculations for tests like t-tests, chi-square, and ANOVA.