ghc-code-review-with-copilot

Evaluates GitHub Copilot code review findings against a planted-issue diff to compute precision and recall.

2|Updated Jun 1, 2026
One-click install
npx skills add https://github.com/jay-steenbergen/MSSAMentorAgent --skill ghc-code-review-with-copilot-jay-steenbergen
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: ghc-code-review-with-copilot
Source: https://github.com/jay-steenbergen/MSSAMentorAgent/tree/main/.github/skills/tracks/github-copilot/ghc-code-review-with-copilot
Command: npx skills add https://github.com/jay-steenbergen/MSSAMentorAgent --skill ghc-code-review-with-copilot-jay-steenbergen

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Engineers adopting AI code review lack a calibrated sense of what Copilot actually catches versus misses, leading to blind trust or blanket dismissal of its output. ## Core Features & Use Cases - Guided flawed-diff exercise: Builds a Python branch with 8 planted issues (SQL injection, MD5 hashing, missing authorization, unsigned JWT, dead code) and opens it as a pull request. - Finding classification workflow: Walks the learner through labeling each Copilot review comment as true positive, false positive, nit, or missed. - Scorecard computation: Calculates precision, recall, and signal density from the tally, then contrasts AI strengths (pattern-match issues) with human strengths (business logic, architecture fit). - Use Case: A learner requests Copilot review on the PR via /review in VS Code Chat or the github.com reviewer panel, tallies results against the answer key, and writes three review comments Copilot missed. ## Quick Start Ask the mentor to start the Copilot code review project and set up the flawed auth.py diff on a throwaway repository.

Frequently Asked Questions about ghc-code-review-with-copilot

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run GitHub Copilot code review on a pull request?

Open the PR on github.com and request a review from Copilot in the Reviewers panel, or use the /review command in VS Code Chat with the file open. Copilot posts findings as inline comments on the diff within about a minute.

What kinds of issues does Copilot code review catch reliably?

Copilot reliably catches pattern-match issues like SQL injection via string concatenation, MD5 password hashing, unsigned JWTs, unreachable code, and missing docstrings. It usually misses context-dependent issues like missing authorization checks and unbounded pagination parameters.

How do I measure precision and recall for an AI code review tool?

Plant a known set of issues in a diff, run the review, then classify each finding as true positive or false positive. Precision is true positives divided by all findings; recall is planted issues caught divided by total planted issues.

Should Copilot code review be used to block pull request merges?

No, the false-positive rate is too high to gate merges. Use it as a first-pass filter for style, dead code, and common security patterns while humans focus on architecture fit and business logic.

What are the limitations of AI code review compared to human review?

AI review lacks business context, so it misses authorization flaws, caller-behavior reasoning like unbounded per_page values, and architecture concerns such as mixed concerns in one module. Humans still win on who-can-do-what reasoning and structural design feedback.