experiment-runner

To run ML evaluation experiments with checkpoint recovery, trigger a run by specifying the dataset benchmark, mode, and model back-end. The system validates intent with a dry run, launches the job, and automatically restarts from the last checkpoint after failures like rate limiting or proxy outages.

Monitoring long-running ML jobs requires real-time streaming or periodic loops with explicit monitor arguments. To prevent config drift, the system performs strict plan conformance verification before and during the run, ensuring ablation series deltas remain trustworthy across phases.

Yes, you can guide an ablation series by specifying the benchmark, mode, and expected run size. The workflow validates intent, executes a dry run, launches each phase with active monitoring, and runs post-completion validation to ensure reliable deltas across different model back-ends.

Checkpoint recovery activates when a long-running evaluation job hits a 429 rate limit or proxy outage. The system detects the failure, stops the broken process, and automatically restarts the evaluation job from the last saved checkpoint, preventing lost progress and wasted compute.

No, this workflow strictly excludes analysis-only or code-modification workflows. It is dedicated to running and rerunning ML experiments, babysitting long-running evaluation jobs, and verifying post-run result validity, rather than modifying experiment code or performing data analysis.

You should not use an automated experiment runner if your goal is code modification or post-run data analysis. The workflow is strictly scoped to launching, monitoring, and recovering ML evaluation runs, handling failures like stale logs or dead processes, and verifying plan conformance.