What problem does it solve? Evaluating whether an AI skill actually improves on the bare model, whether each section earns its place, and whether its description triggers correctly is hard to do rigorously by hand. This Skill runs a skill's eval cases in clean isolated working directories and reports evidence-backed results instead of impressions. ## Core Features & Use Cases - Four eval modes: baseline (skill vs bare model), variant (full vs stripped version), quality (output vs rubric with a read-only grader), and trigger (description firing measured through a platform adapter). - Platform-agnostic adapter seam: runtime-specific invocation, auth, and transcript handling live in a JSON adapter config, with a working Claude Code adapter included. - Variance benchmarking and description optimization: repeat runs are aggregated into mean/stddev/delta statistics, and trigger mode can iterate on a description against a blinded held-out test set. - Use Case: After editing a skill's description, run trigger mode to measure whether it fires on near-miss queries it should catch and stays quiet on the rest, then pick the winning description by held-out test score. ## Quick Start Ask the agent to run the evals for a skill by pointing it at the skill directory, for example: run baseline and quality evals for the skill at .agents/skills/my-skill and report the results.
