Run RED-GREEN experiments to quantify skill effects on agent behavior.
npx skills add https://github.com/jota-batuta/batuta-dots --skill skill-eval-jota-batuta
Please help me install this Agent Skill. Skill: skill-eval Source: https://github.com/jota-batuta/batuta-dots/tree/main/BatutaClaude/skills/skill-eval Command: npx skills add https://github.com/jota-batuta/batuta-dots --skill skill-eval-jota-batuta
This evaluation framework provides a structured, repeatable approach to verify that Batuta skills meaningfully influence agent behavior by running controlled RED-GREEN experiments and reporting outcomes.
Run /skill:eval <skill-name> to validate how the skill changes agent behavior against its eval cases.
High-intent search queries and answers about installing and using this skill.
To evaluate agent behavior changes with a RED-GREEN benchmark, execute a RED baseline run and a GREEN skill-loaded run against predefined eval cases, then compare the results to quantify the skill's impact. This framework outputs measurable verdicts for analysis.
A SKILL.eval.yaml file defines evaluation cases, observable quality metrics, and anti_criteria for skill validation. It provides the structured baseline required to measure how a loaded skill alters agent behavior during automated testing.
You can benchmark a single skill against predefined cases by running the /skill:eval workflow. This executes controlled experiments comparing baseline behavior against skill-loaded behavior and produces a structured envelope with PASS, FAIL, or PARTIAL verdicts.
Yes, you can integrate skill evaluation results into a CI pipeline by parsing the structured envelope with verdicts and reports generated by the evaluation runs. These automated outputs are designed for both CI integration and human review.
The skill:eval workflow validates how a single skill changes agent behavior against its eval cases, while skill:benchmark performs cross-skill benchmarking to measure overall ecosystem health across defined cases.
yu13130122297Identify uncertainties in research framing, datasets, metrics, and baselines.
mkrlabsGenerate structured QA verdicts and metrics for validation workflows.
seanGSISGTest skills with subagents to verify resistance to rationalization under pressure.
Donaldshen27Enforce a red-green-refactor workflow by writing failing tests before code.
