skills.rest
haomingz
haomingzCOMMUNITY
Explore all skills by this author →
GitHub

split-test-evaluator

Analyze AB test data with Z-tests and chi-square tests for statistical significance.

12|2|Updated Apr 21, 2026
Search Tags:#analysis#statistical#power-analysis#ab-test#confidence-interval#chi-square#z-test
One-click install
npx skills add https://github.com/haomingz/kimi-skills --skill split-test-evaluator
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: split-test-evaluator
Source: https://github.com/haomingz/kimi-skills/tree/main/skills/split-test-evaluator
Command: npx skills add https://github.com/haomingz/kimi-skills --skill split-test-evaluator

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

Analyzes AB test data to quantify conversion differences and determine statistical significance, turning raw counts into actionable insights.

Core Features & Use Cases

  • Z-test and chi-square tests for significance of conversion rate differences across variants.
  • Confidence intervals and power analysis to assess reliability and required sample sizes for future tests.
  • Use Case: Product teams evaluating new features or experiments with binary outcomes to decide deployment.

Quick Start

Provide AB test data in JSON format to obtain a complete statistical analysis and recommendations.

Frequently Asked Questions about split-test-evaluator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I calculate statistical significance for my AB test data?▼

To calculate statistical significance for AB test data, you apply Z-test and chi-square tests to binary conversion outcomes, quantifying variant differences and providing clear deployment decisions. It processes raw counts directly to output actionable significance results.

Can I run a power analysis and sample size estimation for small sample AB tests?▼

Yes, you can run power analysis and sample size estimation for AB tests with small or large sample sizes. The analysis assesses result reliability and calculates the required future sample sizes to confidently detect conversion rate differences.

What is the best way to evaluate binary conversion outcomes across experiment variants?▼

The best way to evaluate binary conversion outcomes is applying Z-test and chi-square tests to quantify variant differences. This approach calculates confidence intervals and statistical significance to help product teams decide on new feature deployment.

How do I format my experiment data to analyze conversion rate differences and confidence intervals?▼

To analyze conversion rate differences and confidence intervals, format your experiment data as JSON inputs containing raw binary counts. The analysis processes this JSON format to output statistical significance, confidence intervals, and actionable recommendations.

Related Skills

Explore More Data & Analytics Skills→
spandiosspandios

ab-test-analysis

Analyze A/B test results for statistical significance and actionable recommendations.

Community
Advanced
omeragaakbasomeragaakbas

ab-test-analysis

Analyze A/B test results to compute statistical significance and practical impact.

Community
Intermediate
zm2231zm2231

hypothesis-testing

Design and analyze hypothesis tests for experiments and A/B tests.

Community
Advanced
dkyazzentwatwadkyazzentwatwa
86

ab-test-calculator

Calculate statistical significance for A/B tests using Chi-square, Z-test, or T-test.

Community
Intermediate