Automate agent performance evaluation with Harbor framework tasks.
npx skills add https://github.com/KaiserWhoLearns/skillsbench --skill harbor
Please help me install this Agent Skill. Skill: harbor Source: https://github.com/KaiserWhoLearns/skillsbench/tree/main/.claude/skills/harbor Command: npx skills add https://github.com/KaiserWhoLearns/skillsbench --skill harbor
Harbor provides a standardized framework to validate and run task-based benchmarks for agent evaluation, simplifying task orchestration, execution, and result verification across different models and environments.
Install Harbor, clone the repository, and run an oracle task to validate the setup.
High-intent search queries and answers about installing and using this skill.
Agent evaluation benchmarks are automated by orchestrating task execution and result verification using the Harbor framework across multiple environments, agents, and models to ensure reproducible performance comparisons.
To validate agent tasks before a full benchmark, use the 'harbor tasks check' command to verify task formats under the tasks/, environment/, tests/, and solution/ directories, ensuring proper setup and oracle validation.
To compare multiple AI agents in end-to-end evaluations, use the 'harbor run' command to orchestrate task-based benchmarks, which manages datasets, environments, and results for repeatable performance testing across models.
Yes, you must install Harbor separately via the uv tool or pip to run agent-based testing workflows, as the framework provides the core orchestration, execution, and verification logic required for the benchmarks.
Yes, you can manage datasets and environments for reproducible ML testing within Harbor, which provides standardized task formats and result management to validate and run task-based benchmarks consistently.
Task orchestration for benchmarks across different models fails when task formats under tasks/, environment/, tests/, and solution/ are misconfigured or fail oracle validation during the 'harbor tasks check' phase.
0xDarkMatterAutomate Python virtual environment creation and dependency management with uv.
flashinfer-aiExtract GPU kernel schemas from SGLang models into deduplicated Definition JSON files.
ruvnetOrchestrate distributed agent swarms via MCP tools and CLI commands.
qiushiyanAutomate creation and management of AI agent evaluation tasks in Harbor.
