What problem does it solve? Teams routinely misuse quantitative evidence: they run underpowered A/B tests, peek at results until p < .05, confuse statistical with practical significance, justify redesigns from analytics drop-offs alone, and report fake percentages from 5-user usability studies. This Skill encodes the decision rules that keep quantitative UX claims honest. ## Core Features & Use Cases - A/B test gating and design: A Step 0 decision tree identifies when A/B testing is the wrong tool (why-questions, low traffic, strategic redesigns), followed by a pre-registered hypothesis template, design checklist, and a Lehr's-rule sample-size feasibility gate. - Honest results interpretation: Distinguishes statistical from practical significance using effect size, confidence intervals, guardrail metrics, and cost of change, with fixed-horizon run rules against peeking and metric shopping. - Analytics and small-N firewalls: Enforces a mandatory follow-up qualitative spec before any analytics-driven design change, and bans percentages or averages from small qualitative studies in favor of counts ("3 of 5 participants"). - Use Case: A stakeholder proposes A/B testing a booking-flow change on a low-traffic site. The Skill computes that the test needs 8+ weeks at current traffic, declares it infeasible, and routes the decision to qualitative evidence plus a before/after read with explicit caveats. ## Quick Start Ask the AI to evaluate whether you should A/B test a proposed design change given your traffic and baseline conversion rate, and to write the experiment spec.
