benchmark-suite

Multi-lane benchmark orchestration coordinates an isolated lane matrix across providers and models through a governed cycle of implement-plan, machine verification, and review-cycle with KPI capture. You provide a config JSON to spin up lanes, run them, and generate a final summary.

KPI capture for a multi-provider benchmark suite records metrics, artifacts, and summaries for each lane and the entire run. The workflow supervises the isolated lanes through a governed cycle and persists the captured KPI data into the Brain.

Yes, you can run a 3-provider, 4-model benchmark matrix with per-lane isolation. The suite orchestrates the end-to-end benchmark lifecycle by validating inputs, loading instructions into context, and enforcing safety and execution rules across the isolated lanes.

To start a benchmark suite and run a governed review cycle, you need a valid config JSON. The system uses this configuration to spin up isolated lanes, execute the implement-plan to machine verification workflow, and enforce required execution rules.

Machine verification is required before the review-cycle to validate inputs and ensure execution accuracy across the provider and model matrix. This governed cycle enforces safety rules, controls lane isolation, and guarantees reliable KPI capture before final review.

Yes, benchmarking across a provider and model matrix supports Brain persistence for artifacts. The workflow captures KPIs and summaries for each lane and the whole suite, persisting these artifacts through the Brain during the governed execution cycle.