pdf

To evaluate pilot experiment results, you compute metric summaries across pilot result seeds and compare them against the success criteria defined in the pilot spec to classify outcomes as pass, fail, or inconclusive.

You can automate research wiki updates by enforcing forward-only lifecycle updates that write the derived pilot_result, set the failure_reason with a mandatory prefix, and transition the status field to failed when appropriate.

A pilot verdict report is a persistent record saved to the experiments directory that logs evaluation outcomes. It is generated by reading pilot logs and seeds, then appending traceability entries to the wiki log.

Yes, you can use experiment metrics to enforce forward-only status transitions in an ideation workflow by applying pilot spec success criteria and baseline comparisons to automatically transition wiki states.

When a pilot evaluation yields a fail verdict, the failure_reason field is set in the research wiki with a mandatory [pilot] prefix, and the idea status transitions to failed to maintain traceability.

Classifying pilot outcomes as inconclusive requires reading pilot result JSONs, pilot logs, and the pilot spec to compute metric summaries and compare them against the defined success criterion and baselines.