skill-testing-framework

Automate end-to-end skill testing by generating test templates, running test suites, and comparing execution results against baselines to catch regressions. This framework provides scripts to validate that outputs match expected results.

Run regression tests for skill updates by executing test suites via run_tests.py and validating the outputs against baselines. This ensures changes do not break existing skill workflows or alter expected results.

Yes, you can generate test templates for unit and integration testing of new skills. The framework provides generate_test_template.py to create customizable templates that streamline quality assurance for new skill development.

The skill testing framework requires pyyaml as a dependency for parsing configuration files. You need a Python environment with pyyaml installed to execute the test generation, execution, and validation scripts.

Unit testing validates individual skill components, integration testing checks end-to-end workflows, and regression testing ensures outputs match baselines after updates. The framework supports all three levels to ensure comprehensive skill quality.

Skill test results fail to match baselines when updates introduce regressions or alter outputs. Use validate_test_results.py to compare execution results against baselines, identify discrepancies, and catch unintended changes in skill behavior.