What problem does it solve? It answers whether a skill actually earns its place by measuring it against the bare model, a stripped variant, a rubric, and real trigger queries, instead of relying on intuition about whether the skill helps. ## Core Features & Use Cases - Four eval modes: baseline (skill vs bare model), variant (full vs stripped version), quality (output vs rubric via a read-only LLM grader), and trigger (does the description fire on near-miss queries). - Isolated, platform-agnostic runs: each case executes in a clean working directory with a from-scratch environment, with all runtime specifics behind a JSON adapter seam. - Variance benchmarking and description optimization: repeat runs are aggregated into mean/stddev/delta statistics, and a blinded train/test loop rewrites trigger descriptions without overfitting. - Use Case: Before shipping a new skill, run baseline mode to confirm it beats the bare model, quality mode to grade its outputs against a rubric, and trigger mode to verify its description fires on the right queries. ## Quick Start Run the evals for the skill at path/to/my-skill in baseline and quality mode and tell me where the run folder landed.
