experiment-bridge

To convert an ML experiment plan into executable GPU scripts, the tool reads EXPERIMENT_PLAN.md or FINAL_PROPOSAL.md files and generates training and evaluation code with configurable argparse and reproducible seeds. It matches defined datasets, splits, metrics, and hyperparameters, then validates the implementation with a sanity run before full deployment.

Automating ML training pipeline deployment from a markdown plan involves parsing the plan file to extract milestones like sanity, baseline, main, and ablation stages. The system implements missing scripts, executes a fast sanity check first to validate the setup, and then orchestrates the full deployment while monitoring progress and collecting parseable JSON and CSV metrics for downstream review.

Yes, you can run a sanity check before full GPU experiments by implementing a sanity-first execution strategy. The system executes a quick sanity-stage experiment to validate the training and evaluation scripts, optionally performs cross-model code review, and uses auto-debug-retries to catch errors early, significantly reducing wasted GPU time before launching the main suite.

Experiment implementation requires predefined datasets, splits, metrics, hyperparameters, and success criteria specified in plan artifacts like EXPERIMENT_PLAN.md or FINAL_PROPOSAL.md. The generated scripts must match these exact specifications to ensure reproducible results, parseable JSON/CSV outputs, and proper integration with downstream experiment trackers and review loops.

To collect parseable JSON and CSV metrics from automated ML training runs, the deployment process monitors experiment progress and extracts metrics directly from the execution output. The system updates experiment trackers and logs automatically, producing structured JSON and CSV files that feed into the next review loop for analysis against your predefined success criteria.

Auto-debug-retries in ML experiment deployment are limited to resolving code implementation errors caught during sanity runs or cross-model code review. They cannot fix fundamental issues with the experiment plan itself, such as poorly defined hyperparameters, incompatible dataset splits, or missing success criteria in the EXPERIMENT_PLAN.md or FINAL_PROPOSAL.md artifacts.