test-orchestrator

You can automate QA testing for ClaudeKit workflows by scanning commands, agents, and skills to generate test scenarios. The test orchestrator parses frontmatter to detect executable components and produces structured test data for end-to-end validation.

The best way to generate test scenarios is using an extensible Python script to parse frontmatter and detect executable components. This scans your commands and agents to automatically build a comprehensive, configurable test catalog.

Yes, you can use this to generate structured test data for CI workflows. It automates the discovery and validation of components, producing reproducible step-by-step execution tracks that integrate directly into CI pipelines.

Yes, the test orchestrator supports manual verification checkpoints. It enables step-by-step execution tracks for QA, allowing you to manually validate specific execution points during end-to-end and integration tests.

To scan ClaudeKit components and build a test catalog, run the test orchestrator script. It automatically discovers commands, agents, and skills by parsing their frontmatter and detecting executable components to generate test scenarios.

A limitation is that while it automates regression coverage by generating test scenarios, the workflow requires manual verification checkpoints for end-to-end validation. It relies on parsing frontmatter, so components lacking proper metadata may be missed.