What problem does it solve? It answers whether a skill actually earns its place by measuring it against the bare model, a stripped variant, a rubric, and real trigger queries, instead of relying on intuition about whether the skill helps. ## Core Features & Use Cases - Four eval modes: baseline (skill vs bare model), variant (full vs stripped version), quality (output vs rubric via a read-only grader), and trigger (does the description fire on near-miss queries). - Isolated, reproducible runs: each case executes in a clean working directory with a from-scratch environment, permanent run folders, and timing/token capture written to disk immediately. - Description optimization loop: stratified train/test query splits with the test set blinded from the improver, so trigger descriptions are tuned without overfitting. - Use Case: After editing a skill's description, run trigger mode to measure false positives and false negatives on near-miss queries, then iterate the description across bounded rounds and pick the winner by held-out test score. ## Quick Start Ask the AI to run the evals for the skill at a given path in baseline and quality mode and report where the run folder was written.
