experiment-metrics

Select trustworthy A/B testing metrics by evaluating candidates against the STEDII framework, which scores sensitivity, timeliness, efficiency, debuggability, interpretability, and isolation to ensure metric validity and guide data-driven decisions.

Guardrail metrics protect against unintended business harm during experiments. You should define one primary metric and 3-5 guardrail metrics, governed by a structured decision framework to validate readiness before running product experiments.

Estimate sample size and run pre-experiment checks by conducting A/A sanity tests, assessing metric variance, and planning segmentation. These statistical power checks validate metric reliability before launching your experiment.

The STEDII framework is a six-dimension rubric—Sensitive, Timely, Efficient, Debuggable, Interpretable, and Isolated—used to score candidate metrics during experiment planning, ensuring selected metrics are valid, reliable, and measurable.

You need a metrics framework when planning product experiments to score candidate metrics, define primary and guardrail metrics, run pre-experiment variance assessments, and establish governance to ensure trustworthy data-driven decisions.