ds-experiment-design

What problem does it solve?

It prevents broken or uninterpretable A/B test results by forcing clear causal setup for hypothesis, randomization, metrics, guardrails, and decision rules before analysis begins.

Core Features & Use Cases

• Causal-first experiment specification: Aligns treatment, exposure, observation, interference risks, and the metrics that should move when the business outcome improves.
• Correct unit alignment checks: Validates that the randomization unit, exposure unit, and metric denominator match so results aren’t biased by aggregation mistakes.
• Guardrails and failure-mode planning: Identifies leakage, interference, denominator drift, missing data, novelty effects, and operational constraints that can invalidate conclusions.
• Decision rule readiness: Defines effect size targets, confidence/p-value policy, minimum detectable effect, and practical significance so stakeholders can act on outcomes.

Quick Start

Use ds-experiment-design to create a complete experiment plan for an A/B test of a pricing or ranking change, including hypothesis, unit of randomization, primary/guardrail metrics, key failure modes, and a clear decision rule.