validate-skill-tests

Validate promptfoo test coverage by auditing modified skills for corresponding datasets, valid fixture references, and robust assertions. The check generates a report outlining gaps and recommended test additions.

Check promptfoo fixture references by verifying that paths in tests/promptfoo/fixtures/ point to existing fixtures. This ensures modified skills trigger appropriate tests without broken references.

Assess new behavior coverage by checking that promptfoo assertions cover new paths when a skill changes. This ensures test datasets under tests/promptfoo/datasets validate the updated logic.

No, promptfoo coverage validation does not execute code. It audits the test structure by checking for datasets, fixture references, and assertions, then generates a findings report of gaps.

Identify modified plugin skills by locating SKILL.md changes and extracting skill names. This scopes the validation to skills under plugins/sdlc-utilities/skills/ that require test datasets.

Enforce promptfoo test structure by auditing modified skills for corresponding datasets, valid fixture references, and comprehensive assertion checks. The output describes gaps and guides test additions.