eval-recipes Runner Skill

Eval-recipes benchmarking runs standardized tests on agent implementations to measure performance. Clone the eval-recipes repository, configure your amplihack agent, install dependencies with uv, select a task (linkedin_drafting, email_drafting, arxiv_paper_summarizer, or github_docs_extractor), specify trial count, and execute benchmarks to generate comparative score reports against baseline agents.

Yes. PR validation with eval-recipes compares benchmark scores before and after your changes across specified tasks. Checkout your PR branch, run benchmarks with identical trial parameters, then compare output scores to baseline results to verify improvements without manual testing.

Eval-recipes supports comparison against multiple baseline agents including codex and claude_code. The benchmarking framework generates side-by-side score reports showing how your agent modifications perform relative to these established baselines across your chosen tasks.

Eval-recipes includes four targeted evaluation tasks: linkedin_drafting, email_drafting, arxiv_paper_summarizer, and github_docs_extractor. Each task tests specific agent capabilities, and you can run benchmarks on any combination of these with configurable trial counts to measure performance variation.

Yes. Uv is a required dependency for executing eval-recipes benchmarks. The setup process includes installing uv before running benchmark commands, which manages the Python environment and task execution for consistent, reproducible evaluation results.

Agent configuration setup involves copying your agent configs into the eval-recipes environment after cloning the repository. The Skill handles repository cloning, config placement, dependency installation, and branch/PR selection, then runs benchmarks and produces comparative reports with your customized agent setup.