skill-eval

To evaluate AI agent skills for safety and quality, run an automated audit to surface security and structure issues in SKILL.md files and related assets. This provides a unified score and guidance on reliability and cost before deployment.

An automated audit of agent skill directory structures checks for standardized frontmatter like name and description, evaluates security permissions, and validates the organization of scripts, references, and assets.

Yes, you can use skill evaluation as a quality gate in your CI/CD pipeline. Organizations can pre-screen marketplace skills or validate in-house skills to automate safety, reliability, and cost assessments.

You estimate cost and reliability by running an end-to-end evaluation that assesses functional quality and trigger reliability. This tracks the skill lifecycle and provides guidance on deployment implications.

The best way to pre-screen in-house agent skills for security issues is running an automated audit. It surfaces security and structure issues across SKILL.md files and related directories, providing a unified safety score.