skills.rest
AlexanderMattTurner
AlexanderMattTurnerOFFICIAL
Explore all skills by this author →
GitHub

run-ctf

Runs and reports the live-fire breakout CTF that tests sandbox containment against autonomous agents.

64|11|Updated May 24, 2026
Search Tags:#github actions#ai agent#sandbox#ctf#containment#security evaluation#breakout testing
One-click install
npx skills add https://github.com/AlexanderMattTurner/agent-glovebox --skill run-ctf-alexandermattturner
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: run-ctf
Source: https://github.com/AlexanderMattTurner/agent-glovebox/tree/main/.claude/skills/run-ctf
Command: npx skills add https://github.com/AlexanderMattTurner/agent-glovebox --skill run-ctf-alexandermattturner

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Verifying that a sandbox actually contains a hostile autonomous agent requires a live-fire test, and running, judging, and reporting that test involves many dispatch inputs, posture combinations, and verdict nuances that are easy to get wrong. ## Core Features & Use Cases - CTF Dispatch Guidance: Explains every workflow input for the breakout CTF suite, including model selection, monitor, sandbox backend (sbx or kata), auto mode, whitebox framing, and turn budgets. - Verdict Interpretation: Defines what CONTAINED, BREAKOUT, INCONCLUSIVE, and NO VERDICT each claim and, critically, what they do not claim, including posture-specific caveats. - Round Set Reporting: Describes how to read recorded results from the metrics-history branch and report them as a single table with run links, transcripts, and honest limitations. - Use Case: A maintainer wants to check whether the glovebox sandbox holds against a new model, so they dispatch five control-arm rounds, read the recorded verdicts, and report the containment results with transcript links. ## Quick Start Ask the assistant to run the breakout CTF with five rounds on a chosen model and summarize the containment results as a table.

Frequently Asked Questions about run-ctf

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run the breakout CTF against the sandbox?▼

Dispatch the evals.yaml GitHub Actions workflow with suite set to breakout-ctf, choosing inputs like model, ctf_monitor, ctf_sandbox, and turns. A head commit subject containing [breakout-ctf] or the weekly cron on main also triggers it.

What is the difference between the breakout CTF and Control Tower?▼

The breakout CTF tests whether a live agent can escape the production sandbox, while Control Tower measures marginal defense (delta S/delta U) over the LinuxArena environment catalog. They are different harnesses answering different questions, each with its own skill.

What do the CTF verdicts CONTAINED, BREAKOUT, and INCONCLUSIVE mean?▼

CONTAINED means the agent genuinely attempted breakout and every surface held; BREAKOUT means a surface failed and triggers a page and tracking issue; INCONCLUSIVE means the run adjudicated nothing, such as a model refusal, and is not a pass.

Why did my CTF round get cancelled or produce no verdict?▼

A round dies with its ref: deleting the branch or merging its pull request cancels the in-flight run. Also, a turn budget whose backstop overruns the 360-minute job ceiling is cancelled before judging, producing no verdict.

How do I read a CTF transcript from a web session?▼

Use evals/ctf/dump_transcript.py, which prints one greppable line per message from the public .eval file, since the published index.html is a JS viewer a headless session cannot render. It accepts a run id, --url, or --file, with --refusals-only for refused calls.

When should I dispatch a CTF round with only 2 turns?▼

Dispatch 2 turns when the round exists to exercise the harness rather than the sandbox, such as testing a new judge flag or workflow step. Its verdict is truncated by construction, so it proves the wiring works but says nothing about containment.

Related Skills

Explore More Software Engineering Skills→
Matymatyk-businessMatymatyk-business
61

run-deep-swe

Score AI models on the 113-task DeepSWE benchmark via OpenRouter.

Community
Advanced
jeffreykthomasjeffreykthomas

debug

Diagnose container agent failures with structured debugging steps and log locations.

Community
Intermediate
ComposioHQComposioHQ
1.9k

perf-theory-tester

Validate performance hypotheses through controlled experiments with structured output.

Official
Basic
UnsupervisedcomUnsupervisedcom
49

manual_tests

Automates DeepWork hook and rule tests across isolated sub-agents with enforced constraints.

Official
Basic