benchclaw-stage2-simulator-data-acquisition

Simulator data acquisition is automated by executing tasks defined in the stage2_execution_plan.yaml against your simulator environments. This collects data automatically according to the defined plan, reducing manual effort and ensuring the resulting benchmark datasets are real and reproducible.

The stage2_data_collect phase is the data acquisition stage within the BenchClaw benchmarking pipeline. It interacts with specified simulators to collect real data, ensuring data integrity and generating detailed reports and artifacts that allow the entire data collection process to be reproduced.

Reproducibility during benchmark data collection is ensured by generating detailed reports and artifacts alongside the acquired data. By executing a structured stage2_execution_plan.yaml against the simulators, the process produces verifiable outputs that allow you to reliably reproduce the exact data collection workflow.

Yes, a simulator card is required. The Skill needs access to simulator environments and the ability to execute tasks as defined in both the simulator card and the stage2_execution_plan.yaml to successfully automate the data acquisition phase of benchmark construction.

A key limitation is that this automation strictly requires pre-existing access to configured simulator environments and a defined stage2_execution_plan.yaml. Without the ability to execute tasks as defined in the simulator card, the automated data collection process cannot proceed or generate reproducible artifacts.