hypothesis_generation

Use the IF-THEN-BECAUSE format: state the change you make, the measurable outcome you expect, and the reasoning behind it. For example, changing a CTA button color to increase click rate by 10% because higher contrast draws attention.

A hypothesis is falsifiable if it can be proven wrong by data, and measurable if its outcome maps to a concrete metric like click rate or retention. Statements that are always true or lack a quantifiable result are not valid hypotheses.

Use ICE scoring: rate each hypothesis on Impact, Confidence, and Ease from 1 to 10, then average the three scores. Rank hypotheses by score and also consider the risk if the test fails.

A complete plan includes the hypothesis statement, independent and dependent variables, primary and guardrail metrics, test type and split, duration, and an analysis plan specifying the statistical test, confidence level, and minimum detectable effect.

Avoid A/B testing when traffic is too low to reach statistical significance, when the change cannot be isolated to a single variable, or when the hypothesis is not falsifiable. In those cases, use qualitative research or observational analysis instead.