ad-test-designer

Design and analyze A/B and incrementality tests for paid ad campaigns.

2.5k|345|Updated Dec 18, 2025
One-click install
npx skills add https://github.com/aaron-he-zhu/aaron-marketing-skills --skill ad-test-designer
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: ad-test-designer
Source: https://github.com/aaron-he-zhu/aaron-marketing-skills/tree/main/ad/orchestrate/ad-test-designer
Command: npx skills add https://github.com/aaron-he-zhu/aaron-marketing-skills --skill ad-test-designer

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill addresses the need for designing, setting up, and analyzing A/B and incrementality tests for paid advertising campaigns, providing actionable insights and recommendations.

Core Features & Use Cases

  • Experiment Design: Create A/B/n tests for creative/landing pages, incrementality tests, and more.
  • Statistical Interpretation: Analyze test results to determine statistical significance and practical effects.
  • Decision Support: Generate recommendations based on precommitted action rules and guardrails.
  • Use Case: If you're planning a new A/B test for a paid ad campaign, use this Skill to design the test, set up the experiment, and analyze the results to determine if the changes made were effective.

Quick Start

Use the ad-test-designer skill to design an A/B test for two landing-page hero variants. Baseline CVR is 3%, I want to detect a 15% lift. Goal is DR.

Frequently Asked Questions about ad-test-designer

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I design an A/B test for paid ad campaigns?

Designing an A/B test for paid ad campaigns involves generating hypotheses, variant matrices, and sample-size/duration/power plans to structure your experiment and measure effectiveness.

What is ad incrementality testing and when do I need it?

Ad incrementality testing measures the true lift paid ads generate versus a control group. You need it to validate campaign impact and determine if changes produced actual causal effects.

How do I calculate sample size and duration for an A/B test?

Calculate sample size and duration for an A/B test using baseline conversion rates and expected lift. This determines the statistical power needed to detect meaningful campaign optimization effects.

How do I analyze A/B test results for statistical significance?

Analyzing A/B test results for statistical significance involves generating effect and uncertainty readouts to determine if observed differences in advertising variants are mathematically valid.

Can I use A/B testing for landing page optimization?

Yes, you can use A/B testing for landing page optimization by setting up variant matrices for different hero sections. This detects conversion rate lifts against your baseline metrics.

What are precommitted action rules in campaign optimization?

Precommitted action rules in campaign optimization are predefined decision criteria set before testing. They generate owner-approved recommendations to ensure objective choices based on guardrails and test outcomes.