Define A/B test hypotheses, metrics, sample size, and decision criteria.
npx skills add https://github.com/TedTschopp/tedt.org --skill ab-test-setup-tedtschopp
Please help me install this Agent Skill. Skill: ab-test-setup Source: https://github.com/TedTschopp/tedt.org/tree/main/.github/skills/ab-test-setup Command: npx skills add https://github.com/TedTschopp/tedt.org --skill ab-test-setup-tedtschopp
Planning, designing, and executing rigorous A/B tests to verify hypotheses, optimize outcomes, and reduce decision risk in product development.
Outline a complete A/B test plan by specifying the hypothesis, primary metric, sample size, and intended variants.
High-intent search queries and answers about installing and using this skill.
A/B test design requires defining a testable hypothesis, selecting a test type, and determining the primary metric, secondary metrics, guardrails, and target sample size using baseline traffic data to ensure statistical significance and power.
A/B testing compares two variants, A/B/n compares multiple variants, MVT evaluates combinations of page elements, and split URL redirects traffic to different pages, allowing you to choose the appropriate test type based on your experimentation goals.
Calculating target sample size for an experiment involves applying statistical rigor by using baseline data and traffic estimates to determine the required power and significance level, preventing premature data peeking and invalid conclusions.
Multivariate testing, or MVT, should be used instead of a standard A/B test when you need to understand the interaction effects between multiple page elements simultaneously, rather than just comparing two distinct variant versions.
A rigorous A/B test plan requires predefined decision criteria to determine a winner, alongside secondary metrics and guardrail metrics to monitor for unintended negative impacts on user behavior during the experiment.
Setting up an experimentation plan without historical baseline traffic data is challenging because baseline data is required to specify the target sample size, calculate statistical power, and formulate a testable hypothesis accurately.
ajayatwal1105-emersonGenerate A/B and multivariate test plans with hypothesis framing and sample size calculations.
marmikcfcGenerate messaging variants and A/B test plans for LinkedIn and cold email.
autostay-krAnalyze A/B test results for statistical significance and sample size validation.
GuitarAlchemistSimulate multi-armed bandit algorithms with epsilon-greedy, UCB1, and Thompson sampling.
