train-review

Train and evaluate code review agents against captured GitHub PR test cases.

Updated Aug 23, 2026
One-click install
npx skills add https://github.com/thomasttvo/claude-skills --skill train-review
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: train-review
Source: https://github.com/thomasttvo/claude-skills/tree/main/skills/train-review
Command: npx skills add https://github.com/thomasttvo/claude-skills --skill train-review

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Removes manual and biased evaluation of code review agents by automating blinded reviewer runs against real captured PR test cases, ensuring the agent learns to find the same signals a human reviewer recorded without contamination.

Core Features & Use Cases

  • Load and interpret test cases from a local training directory, extracting PR identifiers, comment IDs, adhoc notes, and referenced issues or follow-ups.
  • Resolve comment signals by fetching GitHub review comment bodies and summarizing expected signals, including handling outdated comments by anchoring to the commit state that produced the comment.
  • Spawn isolated reviewer and judge sub-agents via TeamCreate so reviewers run blind, enforce contamination checks, score results into structured JSON, and iterate prompt changes until an 80% weighted-signal pass or iteration limit.
  • Logging and prompt evolution with a specificity check and automatic updates to the review-pr brief, recording iterations in a training log.

Quick Start

Train and evaluate the review-pr agent on all saved test cases and iterate prompts until the reviewer reaches an 80% weighted match or stops after five iterations.

Frequently Asked Questions about train-review

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate training a code review agent using GitHub pull requests?

Automate code review agent training by loading captured PR test cases from a local directory, spawning blind reviewer sub-agents, and evaluating their ability to detect expected human signals without manual bias.

How does automated reviewer training handle outdated GitHub review comments?

Automated reviewer training handles outdated GitHub comments by anchoring to the specific commit state that produced the original comment, ensuring the agent evaluates the correct historical code context.

Can I evaluate code review agents against real PR test cases without data contamination?

Yes, isolated reviewer and judge sub-agents are spawned via TeamCreate to run blind reviews, enforcing strict contamination checks so the reviewer does not see the expected signals beforehand.

What is the best way to iterate and improve a code review agent's prompt accuracy?

Iterate code review agent prompts by applying a specificity check, automatically updating the review brief, and logging iterations until the agent achieves an 80% weighted-signal match or hits the iteration limit.

Does automated code review training require fetching GitHub diffs and comments via APIs?

Yes, automated code review training requires fetching GitHub PR diffs and comment bodies via APIs to resolve expected signals and provide the necessary code context for blind reviewer evaluation.

What are the limitations when evaluating a code review agent with captured PR test cases?

Evaluation stops automatically after five iterations or when an 80% weighted-signal pass rate is reached, limiting endless prompt loops and bounding the automated training session duration.