UKGovernmentBEISOfficial·17 Agent Skills Included

inspect_evals

Build, test, and audit AI model evaluations with automated workflows

Automates the full lifecycle of LLM evaluation benchmarks, from fixing broken evals and reviewing code quality to generating evaluation reports and auditing third-party eval security. Eliminates manual checklist reviews, slow test triage, and risky blind runs of untrusted evaluation code. Guides AI agents through validity checks, test coverage, CI fixes, and release preparation so benchmarks stay accurate and reproducible.

657|419

↗

npx skills add UKGovernmentBEIS/inspect_evals --all -g -y

Available:

Tells the AI agent how to run evaluations with uv, route requests to the correct workflow skills, follow the master review checklist, and respect repository permissions and PR rules.

All Skills in This Repository (17)

Pure Emerald Level Indicators

📦 In Repo

UKGovernmentBEIS

read-eval-logs

List, dump, convert, and view Inspect evaluation logs via Python APIs and CLI.

code-quality-review-all

Audit evaluations under src/inspect_evals and generate results.json and SUMMARY.md.

investigate-dataset

Inspect HuggingFace, CSV, and JSON datasets for schema and quality.

code-quality-fix-all

Plan, validate, and apply code quality fixes from results.json on a dedicated branch.

eval-report-workflow

Generates reproducible LLM evaluation reports with model selection, cost estimation, and results tables.

generate-asset-actions

Classify external assets into priority tiers and generate asset-actions.yaml from ASSETS.yaml.

check-trajectories-workflow

Analyzes agent trajectories from evaluation logs using Inspect Scout scanners.

build-repo-context

Distills GitHub PRs, issues, and review comments into a repository knowledge base.

ensure-test-coverage

Review and create pytest test coverage for Inspect AI evaluations.

eval-quality-workflow

Fix or review a single Inspect AI evaluation against EVALUATION_CHECKLIST.md standards.

eval-validity-review

Review an Inspect AI evaluation's claims, naming, dataset, and scoring validity.

ci-maintenance-workflow

Diagnose and fix failing CI tests, smoke tests, and slow test markers in GitHub Actions workflows.

Official

Advanced

Frequently Asked Questions

FAQPage Schema

How to install inspect_evals?▼

Run `npx skills add UKGovernmentBEIS/inspect_evals --all -g -y` in your terminal to install all workflow skills in this suite globally.

What does inspect_evals do?▼

It is a registry of community-contributed LLM evaluations for the Inspect AI framework, plus agent skills that automate reviewing, fixing, testing, and reporting on those evaluations.

How to check if an AI evaluation is safe to run?▼

Use the security-audit-eval skill, which inspects the eval's code, sandbox setup, and external downloads, then produces a report with a clear safe or unsafe verdict.

Can AI agents fix failing evaluation tests automatically?▼

Yes. The ci-maintenance-workflow skill diagnoses failing tests or smoke tests from a CI link, applies a verified fix, and opens a draft pull request with full validation notes.

Does inspect_evals work with Claude Code?▼

Yes. The AGENTS.md and CLAUDE.md files give Claude Code and compatible agents the exact commands, permissions, and workflow routing needed to work autonomously in this repository.