eval-regression

Name: eval-regression
Availability: InStock
Author: silviopavanetto

Compare working-tree plugin versions against HEAD using parallel evals.

10|1|Updated Mar 19, 2026

Search Tags:#workflow #testing #regression #version-control #eval #skill-creator

Ready to run

npx skills add https://github.com/reidemeister94/development-skills --skill eval-regression

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: eval-regression
Source: https://github.com/reidemeister94/development-skills/tree/main/skills/eval-regression
Command: npx skills add https://github.com/reidemeister94/development-skills --skill eval-regression

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Compare the current (working-tree) version of a plugin/skill against its last committed version to catch regressions before committing. It acts as a focused regression verifier around the skill-creator tooling to surface behavioral changes early.

Core Features & Use Cases

Snapshot and compare the working copy against HEAD to surface behavioral changes.
Validate evals.json presence, offer schema guidance, and generate starter evals when missing.
Run evals in parallel for old vs new versions, produce per-eval transcripts, and aggregate results into a regression report.
Provide a final verdict and a changelist highlighting touched files.

Quick Start

Run eval-regression on a target plugin or skill to start a regression assessment and generate a report.

Frequently Asked Questions about eval-regression

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I catch skill regressions before committing version control changes?▼

Regression testing for skills compares the working-tree version against the last committed version to catch behavioral changes. It snapshots HEAD and runs parallel evals to surface regressions before you commit.

What happens if my evals.json file is missing when running a regression test?▼

If evals.json is missing or empty during a regression test, the process offers schema guidance or auto-generates starter evals for you. This ensures the parallel old vs new version comparison can proceed without manual setup.

How do I run parallel evals to compare old and new plugin versions?▼

Running parallel evals involves comparing the current working copy of a plugin against its last committed version. The process produces per-eval transcripts and aggregates them into a report highlighting regressions and improvements.

What is included in a regression report for workflow testing?▼

A regression report for workflow testing includes changed files, a success or failure verdict, and a clear summary of regressions and improvements. It aggregates per-eval transcripts to highlight exactly what behavioral changes occurred.

Do I need version control setup to validate my plugin evals?▼

Yes, version control setup is required because the regression assessment snapshots HEAD to compare the old committed version against the new working copy. The process relies on a committed baseline to detect behavioral changes.