karpathy-metric-pre

To run an adversarial pre-mortem, provide the primary metric definition, the intended business outcome, what the optimization agent can edit, and how evaluation is currently performed. This generates a structured gaming vector table detailing how the agent can inflate scores without delivering real value.

Proxy divergence occurs when an optimization metric drifts from the real business outcome it represents. An adversarial pre-mortem identifies how an optimizer can game evaluation hooks or measurement edge cases to inflate the metric while delivering less actual durable value.

Design countermeasures by generating a defense plan that includes secondary metrics, holdout scenarios, and a disappearance test. These countermeasures detect brittle improvements where an agent iteratively changes model prompts or scoring rules to exploit the primary evaluation metric.

A disappearance test validates whether metric improvements represent durable value or brittle gaming. It is included in the defense plan alongside secondary metrics and holdout scenarios to periodically test optimization cycles against evaluation contamination and measurement edge cases.

Use red teaming for metric-driven pipelines where an agent iteratively changes behavior, model prompts, scoring rules, or evaluation procedures. It is needed when you must stress-test a metric to uncover how it can be gamed, contaminated, or diverge from real business outcomes.

Yes, you can evaluate scoring rules by generating an adversarial pre-mortem that enumerates concrete ways an optimizer can exploit measurement edge cases. This produces an evaluation diversity plan to run an adversarial and periodic test regimen across optimization cycles.