eval-recipes-runner

Automate eval-recipes benchmarking to quantify amplihack improvements across tasks.

8|4|Updated Mar 9, 2026

Search Tags:#automation #benchmark #pr-review #ci-validation #eval-recipes #amplihack

Ready to run

npx skills add https://github.com/rysweet/amplihack-rs --skill eval-recipes-runner

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: eval-recipes-runner
Source: https://github.com/rysweet/amplihack-rs/tree/main/amplifier-bundle/skills/eval-recipes-runner
Command: npx skills add https://github.com/rysweet/amplihack-rs --skill eval-recipes-runner

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Automates benchmarking workflows to validate amplihack improvements against eval-recipes benchmarks.

Core Features & Use Cases

Run eval-recipes benchmarks for specific branches and tasks to quantify changes.
Compare scores against baseline runs and generate comparative reports.
Coordinate setup of agent configurations and environment for reproducible results.

Quick Start

Run a benchmark for a chosen task and compare results against a baseline.

Frequently Asked Questions about eval-recipes-runner

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I automate benchmarking for pull request validation?▼

Automating benchmarking for pull request validation requires running the eval-recipes suite to quantify improvements across tasks. This Skill orchestrates CI validation and PR evaluation contexts to generate comparative reports against baseline runs.

What tasks are covered by the eval-recipes benchmark suite?▼

The eval-recipes benchmark suite covers tasks including linkedin_drafting, email_drafting, arxiv_paper_summarizer, and github_docs_extractor. These tasks validate agent improvements across different contexts like research and PR review.

Do I need the uv workflow runner to execute eval-recipes benchmarks?▼

Yes, the uv workflow runner is required to orchestrate tasks when running eval-recipes benchmarks. You also need access to the eval-recipes repository and appropriate agent configurations for reproducible results.

How do I compare benchmark scores against a baseline run?▼

Comparing benchmark scores against a baseline run involves executing eval-recipes for specific branches and tasks. The automation generates comparative reports to quantify changes made by amplihack improvements.

Can I use this automation for CI validation of agent configurations?▼

Yes, this automation applies CI validation contexts to evaluate agent configurations. It coordinates environment setup to ensure reproducible benchmarking results across different branches.

What is the best way to quantify improvements in amplihack?▼

Quantifying improvements in amplihack is best achieved by automating benchmarks with the eval-recipes suite. This validates changes across multiple tasks and generates comparative reports against baseline scores.