Analyze A/B test results into ship, extend, or stop decisions.
npx skills add https://github.com/Wcof/PMSkill --skill pm-abtest-wcof
Please help me install this Agent Skill. Skill: pm-abtest Source: https://github.com/Wcof/PMSkill/tree/main/skills/delivery/pm-abtest Command: npx skills add https://github.com/Wcof/PMSkill --skill pm-abtest-wcof
This Skill helps product teams make reliable A/B test decisions instead of guessing from raw conversion numbers. It validates whether an experiment was run correctly, checks if the result is statistically and practically meaningful, and prevents bad launches when SRM, weak sample size, or guardrail regressions make the evidence unreliable.
Ask the pm-abtest skill to analyze your A/B test by providing control and variant sample sizes, conversions, runtime, and the relevant PMContext success threshold and guardrail metrics.
High-intent search queries and answers about installing and using this skill.
A/B test analysis validates experiment validity, checks SRM and sample size, calculates statistical significance and lift, then outputs a ship, extend, or stop decision based on guardrail regressions.
SRM detection checks for sample ratio mismatch in split testing to verify randomization quality. If SRM is present, the experiment evidence is unreliable and launch decisions must be downgraded to prevent bad releases.
Lift analysis calculates conversion rate differences between variants, while 95% confidence intervals estimate the precision of that lift. Both are compared against success thresholds to validate practical significance.
You should downgrade a launch decision when the primary metric improves but guardrail metrics like revenue, retention, or latency regress, indicating the variant harms important product safeguards despite the conversion gain.
Statistical significance testing requires control and variant sample sizes, conversions, and runtime coverage to verify adequate exposure and detect novelty effects before trusting the experiment outcome.
A/B testing applies to pricing, onboarding, activation, and feature rollout experiments by validating conversion changes against documented product metrics and guardrails to ensure reliable launch decisions.
sarahxu0205Design A/B tests with sample size calculations and statistical significance interpretation.
officebeatsAnalyze A/B test results for statistical significance and sample size validation.
yoanbernabeuTrack and optimize Product Hunt launch conversions with attribution and ROI calculations.
amplitudeAnalyze Amplitude MCP experiments and return structured reports with lift, p-values, and confidence intervals.
