internal-eval-score

The tool computes per-case scores against oracle baselines and compares the current run against the baseline to identify regressions in your evaluation pipeline.

Four-quadrant diff reporting categorizes baseline changes by comparing current and previous run outcomes across four distinct quadrants, highlighting newly passed, newly failed, and persistent test cases.

Run the internal-eval score workflow to capture per-case oracle results and safety scores as a baseline snapshot, storing them as JSON artifacts for future cross-run regression analysis.

Yes, built-in A/B-style comparison utilities evaluate changes and verify safety gates across different evaluation runs to assess safety retention in model workflows.

The workflow generates regression artifacts in both JSON and Markdown formats, providing programmatic access to safety scores, baseline diffs, and structured reporting for pipelines.

No, this regression scoring tool operates without external dependencies, providing a self-contained programmatic interface to compute safety scores and baseline diffs within model evaluation workflows.