empirical-prompt-tuning

Empirical prompt tuning improves agent-facing instructions by running blinded fresh subagents against a frozen evaluation design to reveal ambiguities and drive measurable improvements. This process applies to prompts, skills, command workflows, and code-generation prompts.

Blinded evaluation in prompt tuning prevents bias by using fresh executors unaware of scoring details, running separate execution and scoring passes with evidence-backed decision rules. This ensures outputs are judged strictly on rubric criteria rather than executor expectations.

To run empirical prompt tuning, freeze an evaluation design with fixed scenarios and success criteria, deploy a fresh subagent to run against it, record scoring evidence separately, and iterate on the prompts until gains flatten without hold-out regression.

Yes, empirical prompt tuning applies to code-generation prompts, AGENTS sections, command workflows, and skills. Any agent-facing instructions where ambiguous wording limits success are valid targets for blinded evaluation and iterative refinement.

The best way to evaluate prompt quality without executor bias is to apply a frozen experiment plan with blinded executors and separate scoring passes. This approach isolates prompt ambiguities by using evidence-backed rubrics and fresh subagents.

You should stop iterating during empirical prompt tuning when measurable gains flatten out and no hold-out regression occurs. This indicates the agent-facing instructions have reached stable success against the frozen evaluation design and evidence-backed rubrics.