exp-run

Executes wiki-planned ML experiments through deploy, monitor, and result-collection phases.

Updated Jul 15, 2026
One-click install
npx skills add https://github.com/1549171506/shizhengmubangai1 --skill exp-run-1549171506
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: exp-run
Source: https://github.com/1549171506/shizhengmubangai1/tree/main/Desktop/123/empiricalwiki/i18n/en/skills/exp-run
Command: npx skills add https://github.com/1549171506/shizhengmubangai1 --skill exp-run-1549171506

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Running a machine learning experiment involves many error-prone manual steps: writing training code, deploying it to a GPU, monitoring long-running jobs, and collecting multi-seed results. This Skill automates that entire pipeline for experiments already planned in a research wiki, keeping experiment status and logs synchronized. ## Core Features & Use Cases - Three run modes: deploy mode launches a planned experiment and returns immediately, collect mode checks a running experiment and gathers results when finished, and full mode runs all four phases end-to-end for quick local jobs. - Local and remote deployment: run on a local GPU via screen sessions or deploy to a remote server over SSH with GPU selection, code sync, and result pull-back. - Automated code generation and review: generates train.py, config.yaml, run.sh, and requirements.txt from the experiment plan, with an optional LLM code review pass and a mandatory small-scale sanity check. - Use Case: You have an experiment page in wiki/experiments/ with status planned. Run the deploy mode to generate code, sanity-check it, and launch it on a free GPU; hours later, run collect mode to parse seed results, compute mean and standard deviation, and mark the experiment completed. ## Quick Start Run the experiment with slug my-exp by asking the assistant to execute /exp-run my-exp in deploy mode, then later collect results with /exp-run my-exp --collect.

Frequently Asked Questions about exp-run

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run a planned machine learning experiment end-to-end?

Use the full mode with /exp-run <slug> --full to execute all four phases: code preparation, deployment, monitoring, and result collection. This suits short local experiments that finish in minutes, while longer jobs should use deploy mode followed by collect mode.

How do I deploy an experiment to a remote GPU server over SSH?

Pass --env remote after configuring config/server.yaml with your server details. The pipeline checks connectivity, finds a free GPU, syncs code, installs dependencies, and launches the run in a named remote session.

Can I monitor a running experiment without modifying the wiki?

Yes, collect mode on a still-running experiment only prints a progress report with current step, latest metrics, and detected anomalies. Wiki files are not modified until the experiment process has actually finished.

What happens when training shows NaN loss or CUDA out of memory?

The monitoring phase detects NaN loss, inf loss, OOM, and tracebacks in the logs. It attempts one automatic fix, such as resuming from the latest checkpoint with a reduced learning rate or restarting with a smaller batch size.

Why does deploy mode refuse to run my experiment?

Deploy mode only accepts experiments whose wiki page status is planned. If the status is running, you are prompted to use --collect instead; completed or abandoned experiments are refused entirely.

How are multi-seed experiment results aggregated?

Result files are stored as JSON per seed under results/{slug}/seed_N.json. The collection phase parses all seeds, computes mean and standard deviation per metric, and compares them against the baseline to determine the outcome.