Iteratively modify targets and select improvements using numeric metrics or LLM-as-judge evals.
npx skills add https://github.com/davdunc/pai-framework --skill optimize-davdunc
Please help me install this Agent Skill. Skill: Optimize Source: https://github.com/davdunc/pai-framework/tree/main/skills/Optimize Command: npx skills add https://github.com/davdunc/pai-framework --skill optimize-davdunc
This skill eliminates slow, manual trial-and-error when improving performance, quality, or outcomes by running an autonomous hill-climb loop that keeps changes that improve a measurable target and rejects failures.
Ask the optimizer to improve your target skill by running: /optimize --target "~/.claude/skills/Optimize" --max-experiments 10
High-intent search queries and answers about installing and using this skill.
Automate prompt evaluation using an LLM-as-judge binary eval mode that iteratively modifies targets, measures qualitative improvements, and reverts regressions within an experiment budget.
Autonomously reduce code latency by running a hill-climbing loop that modifies the codebase, executes metric commands to measure numeric targets, and selects only changes that yield improvements.
Yes, you can eliminate manual trial-and-error by applying sandboxed target mutation that automatically applies verified improvements, rejects failures, or partially applies changes after verification.
Yes, metric-mode optimization accepts a user-supplied command that outputs a numeric metric to maximize or minimize, driving the autonomous iteration process to continuously improve the target.
Hill-climbing optimization handles regressions by automatically reverting modifications that fail to improve the measured target, ensuring only successful changes are kept within the time or experiment budget.
LLM-as-judge qualitative refinement requires generated binary evaluation criteria and operates within a sandboxed environment, limiting its scope to evaluating text targets rather than executing numeric metric commands.
DrLuggelsEvaluate LLM applications with automated metrics, human feedback, and benchmarking.
EntelligentsiaConvert scattered LLM prompts into versioned, testable templates with regression tests.
ylxaiIteratively adjust code or AI models using metrics or LLM-as-judge evaluations.
thistleknotRun autonomous hill-climbing loops to optimize scalar objectives via scored candidate changes.
