Pilot result evaluation

To evaluate pilot experiment results, this skill reads seed result JSON files and pilot logs, computes metric summaries against baselines, and writes the verdict directly into your wiki idea metadata fields.

Pilot evaluation classifies experiment outcomes by applying success-criterion logic to your metric summaries and pilot log signals, producing a clear pass, fail, or inconclusive verdict for your idea page.

Automating wiki status lifecycle updates requires applying forward-only lifecycle rules to transition idea page fields, setting pilot_result, logging failure_reason, and moving status to failed when criteria are not met.

You can generate a persistent report by running the evaluation against a specific idea slug, which computes metric comparisons and saves a PILOT_VERDICT_REPORT summarizing metrics, log signals, and next-step guidance.

Yes, you can use this for research automation without manual baseline checks because the system automatically loads the pilot spec, computes metric summaries against defined baselines, and applies the success criteria autonomously.

Pilot evaluation results remain inconclusive when the extracted log signals and computed metric summaries do not clearly meet or violate the defined success criteria, preventing a definitive pass or fail wiki update.