ds-metric-validation

What problem does it solve?

Prevents experiment and validation readouts from being distorted by incorrect metric definitions, unit mismatches, denominator drift, missingness issues, or leakage risks that can silently break conclusions.

Core Features & Use Cases

• Minimum metric checks to verify numerator/denominator definitions, filter logic, null handling, and unit alignment before trusting results.
• Escalation triggers that identify when distributional instability, SRM/invariant failures, denominator drift, sign flips, or segment concentration indicate heightened risk.
• Standard and optional robustness checks to quantify outlier contribution, test alternative definitions, and assess sensitivity to missingness and operational/logging artifacts.
• Use case: A metrics team is about to report treatment lift, but observes unstable denominators and heavy tails; this Skill helps validate interpretability and stability before the number reaches stakeholders.

Quick Start

Use ds-metric-validation to validate the metric definition for an upcoming experiment and surface any risks like denominator drift, missingness imbalance, or potential leakage.