remote-slurm-bootstrap

Submit and monitor Slurm benchmark jobs on remote HPC clusters.

3|Updated Apr 21, 2024
One-click install
npx skills add https://github.com/gabrielfruet/.dotfiles --skill remote-slurm-bootstrap
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: remote-slurm-bootstrap
Source: https://github.com/gabrielfruet/.dotfiles/tree/main/pi/agent/skills/remote-slurm-bootstrap
Command: npx skills add https://github.com/gabrielfruet/.dotfiles --skill remote-slurm-bootstrap

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Manually managing Slurm benchmark jobs across remote HPC clusters leads to inconsistent workflows, missed job status updates, un-synced code errors, and wasted SSH round-trips that consume valuable model tokens.

Core Features & Use Cases

  • Cluster-first SSH execution: All job operations run directly on the remote cluster host to avoid local environment mismatches.
  • Automated file syncing: Syncs local code changes to the cluster via git, rsync, or scp before job submission to ensure jobs run the latest version.
  • Token-efficient polling: Uses planned, combined polling for job status, logs, and output files to minimize unnecessary SSH calls. Use case: ML researchers and HPC engineers running large-scale training benchmarks can use this skill to automate the full job lifecycle from submission to troubleshooting without manual intervention.

Quick Start

Use the remote-slurm-bootstrap skill to submit your benchmark training job to the remote Slurm cluster, sync your latest local code changes first, and monitor the job with optimized polling until it completes.

Frequently Asked Questions about remote-slurm-bootstrap

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate Slurm benchmark job submission and monitoring on a remote HPC cluster?

Automating Slurm benchmark jobs on a remote HPC cluster requires cluster-side SSH execution, pre-submission file syncing via git or rsync, and token-efficient polling for job status and logs. This eliminates manual overhead and inconsistent local environments.

Why do my remote Slurm training jobs fail due to un-synced code changes?

Remote Slurm training jobs fail when local code changes are not synced. Automated file syncing via git, rsync, or scp before job submission ensures the cluster executes the latest benchmark versions without manual SSH intervention.

What's the best way to reduce SSH round-trips when monitoring distributed training jobs?

Reducing SSH round-trips when monitoring distributed training jobs involves token-efficient polling cadences. This combines planned status checks, log retrieval, and output file polling to minimize unnecessary connections to the remote cluster.

Can I run large-scale experiment workflows without manual environment setup on the cluster?

You can run large-scale experiment workflows without manual setup by using environment bootstrapping. This enforces cluster-side execution directly on the remote host, avoiding local environment mismatches for machine learning research.

Does this approach support troubleshooting failed benchmark jobs on remote clusters?

Troubleshooting failed benchmark jobs on remote clusters is supported through combined polling. It retrieves job statuses, logs, and output files efficiently, allowing ML researchers to diagnose issues without excessive manual SSH commands.