autoresearch-is-code-metric-optimizer

An optimize and verify loop requires a modifiable artifact, such as code or config, and a deterministic evaluation method to score results. You modify the artifact, run the evaluation, and keep or discard changes based on the measured outcome.

Iterative backtesting applies to prompt and dataset optimization when you have a deterministic evaluation method to score results. You modify the prompt or dataset, run the evaluation, and keep or discard the changes based on the measured score.

A measurable modify-verify iteration workflow is a process for tuning artifacts against a testable benchmark. It involves modifying an artifact, verifying its performance with a deterministic eval, and keeping or discarding the modification based on the result.

Use an automated optimization loop when your task has a deterministic evaluation method to score results. If your task lacks measurable verification and relies on subjective judgment, use a qualitative debate or research-oriented approach instead.

Iterative optimization does not work without a deterministic evaluation method. The workflow requires a measurable verify function to score changes, ensuring you can objectively decide whether to keep or discard each modification.

The best way to structure trading strategy tuning for measurable gains is a modify-verify-keep-or-discard loop. You modify the strategy, backtest it deterministically, and keep the changes only if the backtest shows an improvement.