skills.rest
scitix
scitixOFFICIAL
Explore all skills by this author →
GitHub

abx-harbor-framework

Runs Harbor benchmarks on AgentBox pre-warmed sandbox pools via an environment plugin.

11|Updated Apr 30, 2026
Search Tags:#benchmark#sandbox#terminal-bench#harbor#e2b#agent evaluation#swe-bench
One-click install
npx skills add https://github.com/scitix/Agent-Sandbox --skill abx-harbor-framework-scitix
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: abx-harbor-framework
Source: https://github.com/scitix/Agent-Sandbox/tree/main/plugin/skills/abx-harbor-framework
Command: npx skills add https://github.com/scitix/Agent-Sandbox --skill abx-harbor-framework-scitix

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires harbor, agent-sandbox-harbor, agent-sandbox-e2b.

What problem does it solve? Running Harbor benchmark suites (Terminal-Bench, SWE-bench, custom datasets) normally requires building a sandbox image per task, which dominates run time. This Skill configures Harbor to claim sandboxes from AgentBox pre-warmed pools through the agent-sandbox-harbor environment plugin, so tasks start with one API call instead of an image build. ## Core Features & Use Cases - Pool-backed benchmark runs: Attach Harbor to AgentBox via --environment-import-path agent_sandbox_harbor:AgentSandboxEnvironment with an env file defining endpoints, pool name, cluster, and image prefix. - Image resolution control: Map tasks to pre-built images using AGBX_IMAGE_MAP for datasets like SWE-bench, or rewrite task.toml docker_image values with AGBX_IMAGE_PREFIX and AGBX_IMAGE_TAG. - Failure diagnosis: Inspect failed sandboxes, logs, and environment events to distinguish image-map or registry problems from individual task failures. - Use Case: Reproduce a Terminal-Bench leaderboard number by sizing a pool to 16 idle replicas, pointing Harbor at the AgentBox data plane, and running harbor run -d [email protected] -a oracle -n 16. ## Quick Start Ask the AI to run the Terminal-Bench 2.0 benchmark with the oracle agent at concurrency 16 on an AgentBox pool, generating the agentbox.env file and harbor command for your cluster.

Frequently Asked Questions about abx-harbor-framework

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run Terminal-Bench on AgentBox sandbox pools?▼

Install harbor and agent-sandbox-harbor, create an env file with E2B_API_KEY, E2B_API_URL, E2B_DOMAIN, pool name, cluster ID, and image prefix, then run harbor with --environment-import-path agent_sandbox_harbor:AgentSandboxEnvironment and --env-file pointing at that file.

How do I run SWE-bench with the Harbor framework?▼

SWE-bench tasks have no docker_image in task.toml, so you must mirror or build the images once and write an AGBX_IMAGE_MAP file mapping each task name to its image. Harbor then resolves images verbatim from that map.

Why are my Harbor benchmark tasks being rejected?▼

Tasks are rejected when no image can be resolved: neither an AGBX_IMAGE_MAP entry nor a task.toml docker_image exists. The rejection is deliberate, so check the image map file and image prefix configuration first.

Why do benchmark tasks time out or never connect?▼

Cold heavy images can exceed the default 300s startup or 600s ready timeouts, so raise AGBX_STARTUP_TIMEOUT and AGBX_READY_TIMEOUT. A never-connects symptom usually means an HTTP/HTTPS mismatch, fixable with AGBX_HTTPS=false for plain-HTTP data planes.

Does agent-sandbox-harbor work with agbx_ API keys?▼

Yes, but e2b SDK 2.24 and later reject non-e2b_ keys client-side. Install agent-sandbox-e2b 0.0.4 or later to neutralize that check, and harbor 0.13 or later pulls a new enough e2b to require it.

How many pool replicas do I need for a Harbor run?▼

The concurrency flag -n must match idle Pods in the pool before the run progresses. Check idleReplicas with abx envs <env> pools and scale with abx scale envs <env> pools <pool> --replicas to at least your concurrency value.

Related Skills

Explore More Software Engineering Skills→
viktorbezdekviktorbezdek
10

hosted-agents

Provision sandboxed background coding agents with per-session state isolation.

Community
Advanced
latticeHQlatticeHQ

tbench

Automate Terminal-Bench integration for Lattice agent benchmarking and failure analysis.

Official
Advanced
AIAsysAIAsys
21

aiasys-tool-dev

Standardize AIASys Agent tool development with parameter validation, sandbox execution, and error handling.

Official
Advanced
Syedyasir001Syedyasir001

hosted-agents

Build sandboxed background coding agents with pre-built images, warm pools, and multiplayer sessions.

Community
Advanced