What problem does it solve?
Running a third-party LLM evaluation can execute untrusted code on your machine, fetch uncontrolled external files, or let an agent escape its sandbox. This Skill performs a read-only security audit of an Inspect AI eval and produces a clear safe / safe-with-caveats / unsafe verdict backed by file:line evidence.
Core Features & Use Cases
- Eight-category threat model: Checks host-side code execution, sandbox configuration, external resource fetching, agent affordances and prompts, resource exhaustion, credential exposure, provenance signals, and out-of-scope validity flags.
- Three mandatory must-checks: Every report explicitly answers whether the eval contains malicious code, uncontrolled externally-fetched files, or sandbox-breakout paths to the host.
- Structured Markdown report: Writes a verdict-driven audit report to agent_artefacts/security_audits/<eval_name>/SECURITY_AUDIT_REPORT.md with an evidence table and severity rubric.
- Use Case: Before running an eval found on GitHub, ask for an audit; the Skill reads the repo via the GitHub API without cloning, inspects compose files, prompts, and dependencies, and tells you whether it is safe to run.
Quick Start
Audit the Inspect AI evaluation at this GitHub URL and tell me whether it is safe to run locally.
