experiment

Confirm the selected idea, baseline reference, dataset split, primary metric, and stop condition, then write PLAN.md and CHECKLIST.md. Run a bounded smoke test first, launch the real run with durable logging, validate outputs, and record the result with artifact.record_main_experiment.

A plan should state the research question, null and alternative hypotheses, baseline and comparability contract, code touchpoints, smoke and full-run commands, stop and abandonment conditions, and fallback options. The provided plan and checklist templates define the canonical structure.

Skip it when the baseline gate is unresolved, the idea stage still has open tradeoffs, or the real need is writing or follow-up analysis rather than a main run. Resolve missing preconditions through the decision stage first.

Launch long runs detached through bash_exec, emit throttled single-line JSON progress markers, and follow a wait-and-check cadence of roughly 60s, 120s, 300s, 600s, then 1800s intervals. Kill and relaunch cleanly if the run is wedged or invalid.

Treat suspiciously strong or baseline-identical metrics as a diagnosis trigger, not a success. Stop brute-force retrying, run the smallest discriminative test to separate hypotheses, and verify comparability before claiming any delta.

A run directory with artifact_manifest.json, run_manifest.json, metrics.json, metrics.md, summary.md, exported logs, and a claim-validation record mapping each claim to a verdict of supported, refuted, or inconclusive, plus an explicit next-action decision.