Orchestrate SLURM fine-tuning jobs with torchtune and evaluate with inspect-ai.
npx skills add https://github.com/niznik-dev/cruijff_kit --skill run-experiment
Please help me install this Agent Skill. Skill: Run Experiment Source: https://github.com/niznik-dev/cruijff_kit/tree/main/.claude/skills/run-experiment Command: npx skills add https://github.com/niznik-dev/cruijff_kit --skill run-experiment
Manually submitting multiple SLURM jobs and constantly monitoring their status is tedious and inefficient. This Skill automates the submission of all fine-tuning jobs and provides continuous, automatic monitoring, freeing you from constant oversight and letting AI handle the heavy lifting.
finetune.slurm jobs with appropriate staggering to prevent cache issues.squeue, sacct) to track job states (PENDING, RUNNING, COMPLETED, FAILED) for all your runs.experiment_summary.md updated with job IDs, start/completion times, and current status for each run.High-intent search queries and answers about installing and using this skill.
Automating SLURM job submission involves coordinating fine-tuning jobs with torchtune and evaluation with inspect-ai across all runs defined in experiment_summary.yaml. This Skill orchestrates end-to-end model experimentation by submitting jobs with appropriate staggering, polling SLURM status in real-time, and updating your experiment summary automatically until completion.
Submit all finetune.slurm jobs at once and let automation handle tracking. This Skill enforces dependency order (optimization before evaluation), monitors job states via squeue and sacct, logs comprehensive progress, and updates experiment_summary.md with job IDs and completion timesāfreeing you to focus on analysis.
Yes. This Skill is designed to orchestrate torchtune-based fine-tuning followed by inspect-ai evaluation. It reads experiment_summary.yaml, invokes both optimizer and evaluator modules sequentially, enforces completion of optimization before evaluation begins, and handles errors including partial successes.
No. This Skill automatically stagger-submits finetune.slurm jobs to prevent cache issues, continuously polls SLURM for status updates (PENDING, RUNNING, COMPLETED, FAILED), and logs all progressāeliminating the need for manual oversight.
This Skill handles errors including partial successes and missing artifacts. It logs comprehensive error details to run-experiment.log, maintains state in experiment_summary.md, and ensures dependency order is enforced so incomplete optimization steps prevent evaluation from running.
You need experiment_summary.yaml defining all runs, finetune.slurm job scripts for each run, and a working SLURM environment. This Skill parses the experiment configuration, validates scaffolding, and coordinates torchtune and inspect-ai modulesāno additional setup required.
niznik-devCreate directories, YAML configs, and SLURM scripts for LLM fine-tuning runs.
KirillKlemGenerate structured interpretive reports from ML model validation metrics.
dongzhuoyaoSubmit GPU training jobs to Slurm clusters with preflight validation and monitoring.
yale-som-hpcManage Slurm jobs on the Yale SOM HPC cluster with sbatch, squeue, sacct, scancel, and scontrol.