What problem does it solve? Choosing between coding agents like Claude Code, Aider, or Codex usually relies on anecdotal impressions rather than data. This Skill replaces vibe-based comparisons with reproducible benchmarks that measure pass rate, cost, time, and consistency on your own codebase tasks. ## Core Features & Use Cases - Declarative Task Definitions: Define evaluation tasks in YAML with prompts, target files, pinned commits, and judge criteria (pytest, grep, or LLM-as-judge). - Git Worktree Isolation: Each agent run executes in a fresh git worktree, so agents cannot interfere with each other or corrupt the base repository. - Comparison Reports: Generate tabular reports comparing agents across pass rate, API cost, wall-clock time, and consistency over repeated runs. - Use Case: Before adopting a new coding agent for your team, define 3-5 tasks representing your real workload, run each agent 3 times per task, and use the report to make a data-backed selection decision. ## Quick Start Ask the agent to define a YAML evaluation task for your repository and run agent-eval to compare Claude Code and Aider across three runs, then generate a comparison report.
