Plan and run AutoAgent Harbor benchmark experiments with isolated output directories.
npx skills add https://github.com/Handy369/passto-pi-frame --skill autoagent
Please help me install this Agent Skill. Skill: autoagent Source: https://github.com/Handy369/passto-pi-frame/tree/main/skills/autoagent Command: npx skills add https://github.com/Handy369/passto-pi-frame --skill autoagent
It helps you run reliable AutoAgent Harbor benchmark and experiment loops while keeping infrastructure stable and ensuring results are analyzed and recorded consistently.
Ask for an AutoAgent Harbor benchmark or an A/B test focused on routing/skill evaluation, and tell the agent to update program.md and tasks/ then run and analyze results into a dedicated experiment output folder.
High-intent search queries and answers about installing and using this skill.
A/B testing experiments for routing and skill evaluation require planning via program.md, controlled edits in tasks/, and isolated output recording to an experiment-specific directory. This ensures stable infrastructure and repeatable benchmark iterations without polluting the baseline worktree.
Benchmarking AutoAgent Harbor workflows involves a two-phase handling approach: separating one-time provider/model compatibility bootstrap from repeatable benchmark iterations. You direct changes through program.md and run configuration, then analyze results.tsv, result.json, and trajectory outputs.
Experiment design keeps baseline worktrees clean by directing benchmark results to isolated experiment output directories, such as /Users/handy/autoagent-outputs/<experiment-name>/. It optionally uses a read-only baseline inheritance from .agent/baseline.md.
Yes, a bootstrap phase is required before formal benchmark iterations to establish one-time provider and model compatibility. This two-phase handling separates initial setup from repeatable benchmark loops to ensure results are analyzed consistently.
Yes, you can inherit baseline configurations for A/B testing by using an optional read-only baseline from .agent/baseline.md. The workflow uses README.md and program.md as the source of truth to direct controlled edits in tasks/ and run parameters.
open-haxValidate OpenCode provider and model configurations against authoritative documentation.
hve4638Parse JSON/CSV experiment results and compute mean and standard deviation statistics.
JamesPrialCreate Go benchmarks with b.N loops, sub-benchmarks, and timing control.
StreamPilotOrgScore growth experiments using the ICE framework and update experiment JSON.
