autostar

Structured experimentation automates optimisation by decomposing goals into tracks and verifiers, then running a mutation-evaluate-reflect cycle. It applies to code, prompts, docs, and configurations to drive quantified improvements.

Verifiers for measuring optimisation progress include deterministic, external-tool, llm_judge, hybrid, and human_gate types. These multi-track rubric systems quantify progress across independent tracks before any work begins.

Setting up an iterative optimisation loop requires an onboarding phase to define goals, assign tracks, set constraints, and allocate a budget. The system then creates an initial mission to start the first mutation-evaluate-reflect cycle.

Memory and run artifacts capture hypotheses, step results, and reflections during optimisation. This durable memory enables cross-run learning by retaining context from every attempted improvement across multiple iterations.

The best way to turn vague improvement goals into a repeatable process is using a framework that enforces explicit goal decomposition and strict verifier types. It converts fuzzy quality aspirations into independent tracks within a disciplined loop.

AITune delegation is an optional feature for inference tuning within optimisation tracks. It supports the multi-dimensional track system alongside other verifier types to drive quantified improvements during the experimentation cycle.