Skills
.Work. You
Rest

internal-eval-validate

Community

End-to-end internal-eval flow validation for CI and local runs.

Software Engineering#automation#testing#validation#end-to-end#ci#harness#qa-engineering
Authorpaulingham
Version1.0.0
Installs0

System Documentation

What problem does it solve?

Delivery validation for /harness:internal-eval: proves the baseline → inject → diff → restore → confirm-clean flow end-to-end with a stubbed inner pipeline. Runs locally and in CI; does NOT execute real /harness:pipeline cases.

Core Features & Use Cases

  • Phase A — Baseline: seed 3 deterministic cases, run-suite with the stub manifest empty (all cases pass), capture-baseline, assert pass_rate == 1.00.
  • Phase B — Inject: flip ≥2 cases in the stub manifest to fail, re-run at a new run-id, rewrite failed_build → failed_diff to match oracle-rejection semantics, re-aggregate.
  • Phase C — Diff: run diff-vs-baseline.sh, assert verdict == "EVAL_FAILED", regression_count ≥ 2, and that both flipped case-ids appear in the regressions quadrant.
  • Phase D — Restore: revert the stub manifest, re-run at a third run-id. Byte-equivalence check: shasum agents/code-reviewer.md before and after MUST match (the live harness is never touched).

Quick Start

bash skills/internal-eval/validate/run-validation-sequence.sh [tmp-dir]

Dependency Matrix

Required Modules

None required

Components

Standard package

💻 Claude Code Installation

Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.

Please help me install this Skill:
Name: internal-eval-validate
Download link: https://github.com/paulingham/.claude/archive/main.zip#internal-eval-validate

Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
View Source Repository

Agent Skills Search Helper

Install a tiny helper to your Agent, search and equip skill from 545,000+ vetted skills library on demand.