Analyze agent-generated code changes against human-approved PRs with file and function coverage.
npx skills add https://github.com/wzh4464/claude-skills --skill diff-eval-func
Please help me install this Agent Skill. Skill: diff-eval-func Source: https://github.com/wzh4464/claude-skills/tree/main/diff-eval-func Command: npx skills add https://github.com/wzh4464/claude-skills --skill diff-eval-func
Evaluate agent-generated code changes against a human-approved PR, using deterministic file coverage and function-level coverage analysis to ensure reproducible evaluation of patch quality across repos and languages.
Provide the PR diff, the ground-truth patch, and the repository path to run a deterministic, function-level diff evaluation and generate a structured evaluation report.
High-intent search queries and answers about installing and using this skill.
To evaluate agent-generated code changes against a human-approved PR, use deterministic file coverage and function-level coverage analysis. This approach compares a PR diff with a ground-truth patch to generate a structured, reproducible evaluation report.
Deterministic function-level diff evaluation is an analysis method that identifies which specific functions are modified by code changes. It computes file coverage via bash scripts to ensure consistent, reproducible patch quality results across different runs and repositories.
To run a function-level diff evaluation, trigger the /diff-eval-func command and provide a PR diff, a ground-truth patch, and a repository path or patch file. The system then executes the evaluation and generates a structured report.
Yes, diff evaluation supports dual semantic and data-based analysis. This dual approach provides structured scoring and automated file filtering to comprehensively assess patch quality during PR validation and AI coding tool benchmarking.
Yes, you can use patch evaluation for benchmarking AI coding tools across multiple repositories and programming languages. It performs ground-truth versus generated patch comparison to ensure reproducible evaluation of patch quality.
Deterministic file coverage relies on bash scripts for consistent results, which may require a compatible environment. The evaluation specifically requires a ground-truth patch and a PR diff to function, limiting its use for patches without a baseline reference.
wzh4464Evaluate agent-generated code against ground-truth diffs and handwritten file lists.
Sheldon-92Automate AI evaluation workflows for benchmarking, regression testing, and safety assessment.
klaaayTrigger pre-merge code review using the superpowers-code-reviewer subagent template.
mouayadakelIdentify test scenarios and generate Jest or Vitest test templates.
