dw-skill-eval-run

You can automate skill evaluation by orchestrating test set loading, cross-model execution, and report generation to validate implementations and collect transcripts. It applies standardized test data and environment isolation to ensure repeatable agent executions.

Collecting execution transcripts for model testing is done by running cross-model execution against a standardized test set. The process gathers transcripts from both Claude and non-Claude models, producing an execution_summary.yaml that aggregates the results.

Subagent isolation during test automation ensures safety and repeatability by executing each skill test in a separated environment. This isolation prevents cross-test interference, allowing you to accurately validate skill implementations against standardized test data.

Yes, you can run regression testing for new skills by loading a standardized test set and executing it across multiple models and configurations. The process aggregates the results into an execution summary, validating skill quality and comparing implementations.

Yes, a standardized test set is required to evaluate skill implementations. The evaluation process depends on standardized test data to load test cases, execute them across different models, and aggregate the resulting transcripts into a summary report.