evaluator-optimizer

Iteratively generate, evaluate, and refine an artifact against explicit pass/fail criteria.

3|Updated Jun 15, 2026
One-click install
npx skills add https://github.com/patrickserrano/lacquer --skill evaluator-optimizer-patrickserrano
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: evaluator-optimizer
Source: https://github.com/patrickserrano/lacquer/tree/main/core/skills/evaluator-optimizer
Command: npx skills add https://github.com/patrickserrano/lacquer --skill evaluator-optimizer-patrickserrano

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? One-shot generation often misses the bar when a task has a concrete, checkable standard — a failing test suite, a lint rubric, or stated document requirements. This Skill turns that into a controlled convergence loop: generate a candidate, evaluate it against explicit criteria, feed the specific failure back into the next attempt, and stop when it passes or a round cap is hit. ## Core Features & Use Cases - Objective-first evaluation: Runs real checks (test suites, go vet, swiftlint, builds) as ground truth before falling back to model-as-evaluator for criteria that cannot be mechanically checked. - Separated generate/evaluate/refine roles: The evaluator grades against stated criteria without silently rewriting, keeping feedback a clean signal for the refinement step. - Capped convergence with escalation: Stops after 3-5 rounds and surfaces persistent failure as a signal that criteria, specification, or approach is wrong rather than spinning indefinitely. - Use Case: A bug fix must pass a test suite — generate the fix, run the tests, feed the exact failure output into the next attempt, and repeat until green or the cap triggers a rethink. ## Quick Start Ask the AI to fix the failing test by iterating with the evaluator-optimizer loop: generate a fix, run the test suite, and refine from the failure output until it passes or four rounds elapse.

Frequently Asked Questions about evaluator-optimizer

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I iterate on code until tests pass with an AI?▼

Use a generate-evaluate-refine loop: produce a candidate fix, run the actual test suite as the evaluator, and feed the concrete failure output back into the next generation. Cap the loop at 3-5 rounds and stop to reassess if it has not converged.

When should I use an evaluator-optimizer loop versus a single attempt?▼

Use the loop only when two conditions hold: there is a clear checkable bar (tests, lint rules, stated requirements) and refinement demonstrably improves on the first draft. If either is missing, iteration just burns rounds without gain.

Should a model or a test suite evaluate generated code?▼

Prefer the objective check — a test suite, build, or linter — whenever one exists, because it is ground truth. Reserve model-as-evaluator for criteria that cannot be mechanically checked, such as architecture quality or prose clarity.

What is the difference between evaluator-optimizer and advisor-checkpoint?▼

advisor-checkpoint is one strategic consult before committing to an approach. evaluator-optimizer is a convergence loop that refines a single artifact against a concrete, checkable standard over multiple rounds.

What happens if the loop never converges?▼

Stop after the round cap and surface the failure instead of continuing. Persistent failure after several rounds usually means the criteria are wrong, the task is underspecified, or the approach needs to change.