ce-retune

Retunes a skill corpus for a new model through measurement-first baseline mining, adversarial audits, and measured cut passes.

24.7k|2.0k|Updated Oct 9, 2025
One-click install
npx skills add https://github.com/EveryInc/compound-engineering-plugin --skill ce-retune
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: ce-retune
Source: https://github.com/EveryInc/compound-engineering-plugin/tree/main/skills/ce-retune
Command: npx skills add https://github.com/EveryInc/compound-engineering-plugin --skill ce-retune

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

When a skill corpus degrades on a new model, rewriting what looks wrong produces plausible fixes with no way to know whether any of them mattered. This Skill replaces guesswork with a measurement-first protocol: mine the run archive for a baseline, establish a noise floor with A/A runs, register a bar before any change, then cut in attributable passes until the bar clears.

Core Features & Use Cases

  • Archive baseline mining: Extracts per-run traces, token rates, and phase markers from historical runs to build an outcome taxonomy and localize where runs die.
  • Noise floor and registered bar: Runs two identical corpus builds through one harness to measure run-to-run spread, then registers a written bar before any edit exists.
  • Adversarial corpus audit: Dispatches independent proposer and defender agents per unit so cuts require a real search of tests, docs, and git history, with protected categories like security guards and machine-readable strings.
  • Halt taxonomy and cut passes: Classifies ten halt mechanisms (phantom handoffs, fictional seams, turn-ending output rules) and applies one problem class per pass with disjoint file ownership.
  • Use Case: A team ports their 31-skill plugin to a new model and sees runs stalling at one phase boundary. This Skill mines 458 archived runs to localize the defect, proves the noise floor with A/A runs, then lands measured cut passes until eight consecutive clean runs clear the registered bar.

Quick Start

Ask the agent to retune the skills corpus in ./skills for the new target model, starting by checking that a benchmark harness with a build selector and run archive exists.

Frequently Asked Questions about ce-retune

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I retune a skill corpus for a new model?

Mine the run archive for a baseline rate, run A/A tests on two identical builds to establish the noise floor, register a written bar, then audit the corpus adversarially and cut in measured passes. Loop cutting and measuring until the pre-registered bar clears.

What is a noise floor in prompt corpus benchmarking?

A noise floor is the run-to-run variation produced by running two identical corpus builds through one harness. Any later improvement claim smaller than that spread is unsupported, so the floor must be measured before any change is made.

Can I retune a corpus without a benchmark harness?

No. The Skill requires a run archive, a build selector that points runs at a specific corpus checkout, and a repeatable task. Without all three it stops and names what to build, because a static audit alone cannot show whether a cut helped.

Why do AI agent runs stall or halt mid-workflow?

Most halts come from prose written as if a second party were waiting: hand-off verbs, return envelopes, output-only rules, and waits on helpers the runtime already ended. The halt taxonomy in the references classifies ten such mechanisms with grep patterns and replacements.

What should never be cut from a skill corpus during an audit?

Machine-readable strings other units parse, security guards, and platform gotchas where wrong behavior looks like success must survive even without provenance. Tests pinning exact strings are findings to report, never tests to weaken.

How many benchmark runs are needed to detect an improvement?

Plan at least 12 A/A runs to size the noise floor, then use Wilson intervals and Fisher's exact test at small n. A cheaper alternative is a streak of N consecutive clean runs against an independently established baseline, which is an exact test needing no second arm.