evaluation-running

You can automate AI safety evaluations across model generations by running a command that executes evals against a specified model panel and budget, validating instruments, and analyzing results with statistical tests.

Back-testing AI safety evaluations includes statistical tests and trend analysis to validate instruments and identify safety issues across different model generations.

Validating AI safety evaluation instruments is automated by the skill, which checks accuracy and reliability across the specified model panel before executing the full evaluation.

You need access to multiple model generations and evaluation instruments, plus an evaluation ID, model panel, and budget to execute the automated evaluation run.

Yes, you can specify a budget parameter when running evaluations to control resource consumption while still automating execution and statistical analysis across model generations.

Automating AI safety evaluation running enables efficient back-testing across model generations, validating instruments and generating trend analysis that manual testing cannot easily scale to.