eval-implementation

Run parallel A/B handoff sessions and generate a structured final report.

Updated Mar 18, 2026

Search Tags:#automation #cli #evaluation #worktree #handoff #grading #ab-testing

Ready to run

npx skills add https://github.com/brianruggieri/skills --skill eval-implementation

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: eval-implementation
Source: https://github.com/brianruggieri/skills/tree/main/handoff/eval-implementation
Command: npx skills add https://github.com/brianruggieri/skills --skill eval-implementation

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Orchestrates controlled end-to-end A/B handoff experiments by running parallel sessions from a common source branch.

Core Features & Use Cases

Orchestrates two parallel worktrees from a source branch to compare implementation strategies.
Runs compare_implementations.py to generate metrics and diffs.
Dispatches blind graders with a rubric and compiles per-session scores and a final verdict.
Produces an eval state JSON and a final report for audit and reproducibility.

Quick Start

Run the setup phase to initialize two worktrees from the selected source branch, plan file, and handoff document, then run the compare phase to generate the final evaluation report.

Frequently Asked Questions about eval-implementation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I run A/B handoff experiments to compare two implementation strategies?▼

To run A/B handoff experiments, you orchestrate two parallel worktrees from a common source branch, execute compare_implementations.py to generate metrics, and dispatch blind graders to produce a final structured report.

What is blind grading in end-to-end evaluation workflows?▼

Blind grading in end-to-end evaluation dispatches automated graders with a predefined rubric to score parallel implementation sessions without knowing which strategy was applied, ensuring unbiased result aggregation and a final verdict.

How do I manage git worktrees for parallel implementation sessions?▼

Git worktree management for parallel sessions involves initializing two distinct worktrees from a selected source branch, propagating a plan file and handoff document to both, and tracking state via an eval state JSON for reproducibility.

Can I automate the entire evaluation process from setup to final report?▼

You can automate the evaluation process by running a setup phase to initialize worktrees and propagate plans, followed by a compare phase that aggregates blind grading scores and generates a final audit-ready report automatically.

Do I need any specific dependencies to run parallel A/B handoff evaluations?▼

No specific dependencies are required to run parallel A/B handoff evaluations, as the workflow manages worktrees, plan propagation, and report generation natively through its internal orchestration and grading scripts.

When should I use A/B handoff experiments for my software engineering workflow?▼

Use A/B handoff experiments when you need to compare implementation strategies from a common source branch with controlled variables, requiring blind grading and reproducible metrics to determine the superior approach for complex handoffs.