benchmark-reviewer

To validate a benchmark implementation for handoff, perform an evidence-based, read-only review of benchmark scripts, definitions, results, and renders to ensure geometry validity, solvability, and manufacturing constraints. This produces stage-scoped review YAML pairs under the reviews directory.

An evidence-based benchmark review checklist includes geometry validity, solvability, runtime randomization, and exact inventory grounding. It verifies readiness by reviewing benchmark_script.py, benchmark_definition.yaml, validation_results.json, simulation_result.json, scene.json, and related evidence against approved contracts.

Yes, you can review benchmark artifacts without modifying original files because the review process is strictly read-only. It examines benchmark scripts, definitions, validation and simulation results, and renders to produce review YAML outputs without altering the source artifacts.

To ensure benchmark definitions are grounded in the latest revision, review benchmark_definition.yaml and benchmark_assembly_definition.yaml against the latest revision and approved contracts. This enforces exact inventory grounding and validates manufacturing constraints for consistent handoffs.

The best way to verify benchmark solvability and geometry validity is through a reproducible review checklist applied to benchmark_script.py, scene.json, renders, and simulation_result.json. This evidence-backed approach ensures traceable decisions and consistent engineer graph handoffs.

Benchmark validation can fail during the review process when geometry validity, solvability, runtime randomization, or inventory grounding checks do not align with the latest revision and approved contracts. The strict checklist flags these issues before handoff to prevent downstream errors.