skills.rest
mattwg
mattwgCOMMUNITY
Explore all skills by this author →
GitHub

pricing-test-impact-on-promo

Analyzes how an A/B pricing test confounds a concurrent promo using SQL and matplotlib trendlines.

1|1|Updated Jul 31, 2026
Search Tags:#impact analysis#sql#matplotlib#a/b testing#pricing test#promo analysis#epic experiment
One-click install
npx skills add https://github.com/mattwg/eureka-consumer-ds --skill pricing-test-impact-on-promo-mattwg
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: pricing-test-impact-on-promo
Source: https://github.com/mattwg/eureka-consumer-ds/tree/main/skills/data-science/consumer_and_degree_strategy_ds/consumer_ds/consumer_promo_analysis/pricing_test_impact_on_promo
Command: npx skills add https://github.com/mattwg/eureka-consumer-ds --skill pricing-test-impact-on-promo-mattwg

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires pandas, numpy, matplotlib.

What problem does it solve? When an A/B pricing test overlaps with a live promotion, their effects confound each other, making it hard to attribute cash and user impact to either one. This Skill provides a worked methodology — built on the High Income Pricing Test vs Q2 '26 C+ Annual Tentpole Promo case — for isolating the test population, splitting metrics by promo phase, and calculating the final USD impact of the test on the promo. ## Core Features & Use Cases - Test population isolation: SQL queries identify EPIC Test/Control users with impression windows, including caveats about EPIC stickiness where users keep seeing test prices after the test ends. - Phase-split impact tables: Aggregates Cash, Users, and Cash-per-user by arm (Control/Test) across Pre-promo, Early Bird, and Post-Early-Bird periods, with promo/non-promo/total cuts. - Sanity checks and visualizations: 50:50 traffic-split checks by channel and country, urgency-messaging exclusion for apples-to-apples phase comparison, and matplotlib trendline charts (dual-axis and preferred single-axis split versions). - Use Case: A lifecycle marketing promo launches while a pricing experiment is still running. Use this Skill to quantify how much of the promo's cash performance was actually driven by the test, producing a final USD impact figure via the Actual method (Control promo cash minus Test promo cash). ## Quick Start Ask the assistant to analyze the impact of your pricing test on the overlapping promo, providing the experiment ID, test and promo date ranges, early-bird window, and promotion IDs so the SQL queries can be adapted.

Frequently Asked Questions about pricing-test-impact-on-promo

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I measure the impact of a pricing test on a concurrent promo?▼

Isolate the experiment's Test and Control users with their impression windows, then split Cash, Users, and Cash-per-user by promo phase (Pre-promo, Early Bird, Post-Early-Bird). Calculate final impact as Control promo cash minus Test promo cash using the Actual method.

How to check if an A/B test traffic split is 50:50?▼

Count distinct impressed users per arm grouped by L0 marketing channel and by country, not just in aggregate. A channel-level or country-level imbalance can hide inside an overall 50:50 split and bias the analysis.

What is EPIC stickiness and why does it matter for test analysis?▼

EPIC stickiness means experiment users continue seeing test prices even after the test officially ends. The impression end timestamp is a global constant, so Test/Control tagging stops at the test end date while pricing effects persist, requiring careful window selection.

Why exclude urgency-messaging days from promo phase comparisons?▼

The trailing 3-4 days of promo phases carry urgency messaging in banners, which skews transaction metrics. Excluding those days makes Early Bird and Post-Early-Bird periods an apples-to-apples comparison for isolating the test's effect.

What are the limitations of this pricing test impact analysis?▼

The queries hardcode experiment IDs, promotion IDs, and date ranges specific to one case, so they must be adapted per analysis. The CURRENT_DATE - 2 upper bound silently includes post-promo transactions if rerun later, and the methodology is a starting template rather than a turnkey solution.

Related Skills

Explore More Data & Analytics Skills→
openaiopenai
5.3k

airtable-filters

Build Airtable record filters using field IDs, operators, and nested AND/OR logic.

Official
Intermediate
shekerkammashekerkamma

size-opportunity

Quantify financial value and business impact of proposed opportunities.

Community
Intermediate
jonnytacos8jonnytacos8

sanity-check

Validate financial figures and cross-check totals against rag_search context.

Community
Intermediate
HaipingXuHaipingXu
21

causal-inference-analysis

Implement causal inference methods in Python and Stata with diagnostic checks.

Community
Advanced