terminal-bench-loop

Drives Terminal-Bench problems through bounded Paperclip iteration loops with board-gated fixes.

Updated Jul 14, 2026
One-click install
npx skills add https://github.com/Douglas-v/strategic-ai-paperclip-lab --skill terminal-bench-loop-douglas-v
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: terminal-bench-loop
Source: https://github.com/Douglas-v/strategic-ai-paperclip-lab/tree/main/paperclip/skills/terminal-bench-loop
Command: npx skills add https://github.com/Douglas-v/strategic-ai-paperclip-lab --skill terminal-bench-loop-douglas-v

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Running a Terminal-Bench task repeatedly until it passes often devolves into unbounded retries, silent stalls, and untracked code changes. This Skill imposes a bounded, auditable loop: each iteration runs a smoke test against an isolated Paperclip App worktree, diagnoses the exact stop point, and requires board confirmation before any product fix is implemented. ## Core Features & Use Cases - Bounded iteration loop: Runs a single Terminal-Bench task through Paperclip with an explicit iteration budget, wall-clock caps, and defined stop rules (pass, board rejection, budget exhaustion, or named blocker). - Structured issue topology: Organizes the loop as a tree of Paperclip issues — a top-level loop parent, iteration children, diagnosis documents, and board confirmation interactions — wired with blockedByIssueIds dependencies. - Worktree isolation: Pins the benchmark command to an isolated Paperclip App worktree via PAPERCLIPAI_CMD and preserves the Harbor dispatch runner config so every iteration tests the same workspace where fixes land. - Use Case: An engineer is asked to "drive Terminal-Bench fix-git until it passes." The Skill opens a loop issue, runs a smoke, diagnoses that the benchmark issue was never assigned a heartbeat-enabled agent, proposes a product rule, obtains board approval, applies the fix in the same worktree, and reruns until the smoke passes. ## Quick Start Ask the agent to run the Terminal-Bench task fix-git through Paperclip in a bounded loop with an iteration budget of three, diagnosing each stop and requesting board confirmation before any product fix.

Frequently Asked Questions about terminal-bench-loop

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run Terminal-Bench in a loop through Paperclip?

Open a top-level loop issue recording the task name, iteration budget, benchmark command, and worktree pointer, then create one iteration child per run. Each iteration runs a bounded smoke, diagnoses the stop point, and only applies product fixes after board confirmation.

How do I diagnose why a Paperclip agent run stopped?

Walk the issue tree the smoke produced node by node and find the exact issue and status combination that halted progress, quoting run ids and status transitions. Classify each stalled issue as needing human intervention, agent-actionable but unrouted, or already covered.

When should I not use the Terminal-Bench loop workflow?

Do not use it to modify the paperclip-bench harness itself, to submit comparable benchmark results for ranking, or for ordinary Paperclip product bugs unrelated to a loop. Those cases follow normal engineering flow or go to BenchmarkQualityManager.

Why does a Harbor benchmark run create an unassigned issue with no heartbeat?

A bare Harbor command that omits the dispatch runner config boots Paperclip but leaves the benchmark issue unassigned with zero heartbeat-enabled agents. Export the full runner config with assignee, heartbeat strategy, and agent adapter, and treat the bare run as a harness setup failure.

Can the loop apply code fixes without approval?

No. Every product fix must be written as a plan document and accepted through a request_confirmation interaction targeting the board before any implementation child issue is created. Rejected proposals end the loop rather than being silently retried.