auto-research

Automate map quality optimization by mutating the example bank, running online evaluation against the AI API, and keeping only improvements to continuously optimize map generation quality. The system reverts unsuccessful mutations automatically.

An autonomous optimization loop for machine learning experimentation mutates examples, evaluates them online, and keeps improvements while reverting failures. It ensures reproducibility using baselines, train/holdout splits, and mutation logs for full traceability.

Yes, online evaluation requires a running dev server. The workflow applies to online map-generation processes that rely on real AI prompts and a running dev server to test and validate mutations against the live system.

Invariants protect critical system files during example bank mutation. This allows safe exploration and mutation of the example-bank.ts file while preventing unintended modifications to essential system configurations or workflows.

Ensure reproducibility in map generation experiments by using baselines, train/holdout splits, and mutation logs. The workflow logs mutation outcomes for traceability and consistent result verification across runs.