skills.rest
reidemeister94
reidemeister94COMMUNITY
Explore all skills by this author →
𝕏@silviopavanettoGitHub

eval-regression

Compare working-tree plugin versions against HEAD using parallel evals.

10|1|Updated Mar 19, 2026
Search Tags:#workflow#testing#regression#version-control#eval#skill-creator
One-click install
npx skills add https://github.com/reidemeister94/development-skills --skill eval-regression
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: eval-regression
Source: https://github.com/reidemeister94/development-skills/tree/main/skills/eval-regression
Command: npx skills add https://github.com/reidemeister94/development-skills --skill eval-regression

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Compare the current (working-tree) version of a plugin/skill against its last committed version to catch regressions before committing. It acts as a focused regression verifier around the skill-creator tooling to surface behavioral changes early.

Core Features & Use Cases

  • Snapshot and compare the working copy against HEAD to surface behavioral changes.
  • Validate evals.json presence, offer schema guidance, and generate starter evals when missing.
  • Run evals in parallel for old vs new versions, produce per-eval transcripts, and aggregate results into a regression report.
  • Provide a final verdict and a changelist highlighting touched files.

Quick Start

Run eval-regression on a target plugin or skill to start a regression assessment and generate a report.

Frequently Asked Questions about eval-regression

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I catch skill regressions before committing version control changes?▼

Regression testing for skills compares the working-tree version against the last committed version to catch behavioral changes. It snapshots HEAD and runs parallel evals to surface regressions before you commit.

What happens if my evals.json file is missing when running a regression test?▼

If evals.json is missing or empty during a regression test, the process offers schema guidance or auto-generates starter evals for you. This ensures the parallel old vs new version comparison can proceed without manual setup.

How do I run parallel evals to compare old and new plugin versions?▼

Running parallel evals involves comparing the current working copy of a plugin against its last committed version. The process produces per-eval transcripts and aggregates them into a report highlighting regressions and improvements.

What is included in a regression report for workflow testing?▼

A regression report for workflow testing includes changed files, a success or failure verdict, and a clear summary of regressions and improvements. It aggregates per-eval transcripts to highlight exactly what behavioral changes occurred.

Do I need version control setup to validate my plugin evals?▼

Yes, version control setup is required because the regression assessment snapshots HEAD to compare the old committed version against the new working copy. The process relies on a committed baseline to detect behavioral changes.

Related Skills

Explore More Software Engineering Skills→
fivetakufivetaku
114

git-teacher-upload

Push committed Git work to GitHub with beginner-friendly explanations.

Community
Intermediate
arvidfcjarfalla-progarvidfcjarfalla-prog

reviewer

Review only changed files with fresh context and return a structured verdict.

Community
Intermediate
pax-kpax-k
6

skill-creator

Draft, test, and iteratively improve AI skills with SKILL.md and eval sets.

Community
Advanced
sjinkssjinks

wp-plugin-version-diff

Generate unified diff patches between WordPress plugin versions.

Community
Advanced