measurement-critic

Validating measurement instruments requires evaluating construct validity, reliability, and bias across LLM-based designs. Multi-perspective assessments provide statistical, methodological, and domain-expert critiques to ensure your instruments measure the intended constructs accurately and resist manipulation.

Reviewing weighted scoring functions for proxy validity and adversarial robustness prevents Goodhart's Law exposure. This composite-scorer mode evaluates sensitivity and manipulation resistance to ensure your hand-tuned weighted ranking systems measure intended metrics accurately.

Evaluate your automated scoring system before deployment to prevent Goodhart's Law failures. Pre-deployment reviews of measurement designs and scoring functions ensure your metrics measure intended constructs accurately and resist manipulation, avoiding flawed decision-making.

The best way to check composite scoring functions for bias and measurement validity is through multi-perspective assessments. Combining statistical, methodological, and domain-expert critiques evaluates calibration adequacy, proxy validity, and sensitivity to ensure robust measurement designs.

Yes, measurement validation operates across both LLM-based measurement designs and hand-tuned weighted ranking systems. LLM-instrument mode evaluates construct validity and bias, while composite-scorer mode reviews proxy validity, sensitivity, and adversarial robustness for weighted functions.