ce-optimize

Automated optimization experiments are coordinated using a repeatable measurement harness that evaluates code and config variants against a baseline. It requires a valid optimization spec and manages parallel evaluation with crash-safe logging to preserve results.

Git worktree workspace management isolates reproducible experiments by creating separate scratch spaces for each variant. This ensures measurement harnesses run consistently without cross-contamination between parallel optimization evaluations.

Batch evaluation supports LLM-as-judge scores alongside hard metrics. The system includes automatic judge evaluation, requiring optional judge configuration within your optimization spec to score and compare code variants.

You need Python3 installed, a complete optimization spec including name, metric, and measurement definitions, plus a defined mutable and immutable scope. Optional judge configuration and approved dependencies can also be specified.

Defining mutable and immutable scopes isolates which code or config variants the optimization experiments can alter. This constraint ensures the baseline measurement harness remains reproducible across parallel evaluations without unintended side effects.

Experiment logs are preserved through crash-safe logging that persists results directly to disk. This ensures your baseline data and completed batch evaluations survive interruptions during parallel optimization workflows.