Run head-to-head experiments to compare two skills on shared evaluations.
npx skills add https://github.com/patanet7/skillproof --skill skill-compare
Please help me install this Agent Skill. Skill: skill-compare Source: https://github.com/patanet7/skillproof/tree/main/skills/skill-compare Command: npx skills add https://github.com/patanet7/skillproof --skill skill-compare
This skill enables reliable, side-by-side evaluation of two skills on the same tasks to determine which approach delivers better results, reducing guesswork in skill selection.
Initialize the matchup with two skills and an evals.json file, then run the matchup and generate the final report.
High-intent search queries and answers about installing and using this skill.
Benchmarking two skills involves running them on a shared set of evaluations to measure performance differences. The head-to-head matchup executes identical tasks, aggregates optional benchmark data for tiebreakers, and produces a final report indicating the winner.
Blind comparison uses randomization and labeling on the outputs of both skills before judging. This prevents bias when evaluating the head-to-head results across multiple runs and configurations.
To set up a head-to-head matchup, initialize the workflow with the two skills and an evals.json file. After initialization, execute the matchup runs and generate the final summarized report.
Yes, you can benchmark skills across multiple runs and configurations. The process aggregates optional benchmark data to surface quantitative performance differences and uses them as tiebreakers in the final report.
Yes, the evaluation workflow enforces reproducibility by exporting per-eval outputs. It also generates a final matchup report with optional benchmark data to ensure the comparison results can be consistently reviewed.
reidemeister94Compare working-tree plugin versions against HEAD using parallel evals.
fray-cloudDetect performance regressions by comparing JS timing data against baselines.
samuelross99Generate competitor comparison and alternative pages from centralized data templates.
zardusai-cyberCompare coding agents on reproducible YAML-defined tasks with pass rate, cost, time, and consistency metrics.
