validate-skill-tests

What problem does it solve?

This skill helps verify promptfoo test coverage for plugin skills, ensuring that modified skills have appropriate test datasets, valid fixture references, and comprehensive assertion checks.

Core Features & Use Cases

• Identify modified skills by locating SKILL.md changes and extract skill names.
• Validate that testing datasets exist under tests/promptfoo/datasets for each modified skill.
• Check fixture references in tests/promptfoo/fixtures/ and ensure they point to existing fixtures.
• Assess new behavior coverage when a skill changes, ensuring assertions cover new paths.
• Generate a concise findings report outlining gaps and recommended test additions.

Quick Start

Run this check after creating or modifying a plugin skill to confirm promptfoo test coverage.