os-eval-lab-setup

Setting up an isolated evaluation lab involves bootstrapping a standalone repository populated with master plugin files, an eval engine, and ready-to-run workflows. The skill automates scaffolding the lab, configuring git remotes, and copying plugin code for controlled iterative experiments.

Skill evaluation runs iterative test cycles to autonomously optimize plugin performance. An isolated lab repo prevents uncontrolled changes to production code by locking evaluation artifacts, copied plugins, and generated instructions to a controlled environment.

Yes, the os-eval-runner engine is a required dependency to bootstrap the testing environment. The skill relies on this engine alongside copilot and gemini tooling to drive the autonomous evaluation loop within the prepared lab repository.

Yes, the config-driven setup prompts for lab path, plugin path, skill, and round labels to automate scaffolding. It copies plugin code and configures git remotes to prepare repeatable experiments across multiple skills.

Eval instructions are generated automatically from templates into an eval-instructions.md file. This generated file drives the autonomous evaluation loop by providing the iterative testing parameters needed for the agent within the lab environment.

The standalone lab repo confines evaluation artifacts to a controlled environment, meaning tests run in isolation without direct access to the master plugin's live production data. It requires the os-eval-runner engine to execute the eval cycles.