skills.rest
dongzhuoyao
dongzhuoyaoCOMMUNITY
Explore all skills by this author →
Website𝕏@vtaohuGitHub

sue-parallelism-discussion

Validate ML/HPC parallelism plans for GPU-occupancy-first node packing.

Updated Aug 27, 2026
Search Tags:#gpu optimization#experiment workflow#hpc scheduling#parallelism planning#scale-up experiments#slurm job packing#node occupancy
One-click install
npx skills add https://github.com/dongzhuoyao/deepresearch --skill sue-parallelism-discussion
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sue-parallelism-discussion
Source: https://github.com/dongzhuoyao/deepresearch/tree/main/.codex/skills/sue-parallelism-discussion
Command: npx skills add https://github.com/dongzhuoyao/deepresearch --skill sue-parallelism-discussion

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill eliminates wasted GPU resources, failed job submissions, and inconsistent experiment results caused by unplanned parallelism, poor node packing, and unvalidated concurrency strategies for ML/HPC scale-up experiments run on remote sandbox backends like LUMI, Snellius, and RunPod.

Core Features & Use Cases

  • Pre-submission parallelism gate: Acts as a mandatory deliberation step between launcher script writing and dryrun submission, ensuring all parallelism plans are explicit and auditable before any compute resources are consumed.
  • GPU-occupancy-first planning: Enforces rules that pack full nodes with exactly one worker per GPU, only allowing partial final nodes, to maximize hardware utilization and avoid idle GPU waste.
  • Contract validation and visualization: Integrates with SUE family audit tools and Gantt chart generation to validate plans against family-wide contracts and visualize wave scheduling, node allocation, and potential idle GPU periods.
  • Use Case: A researcher preparing to run 20 generation variants on a LUMI sandbox can use this Skill to calculate the optimal max_parallel value, node packing shape, and wave schedule that avoids queue limit failures and ensures all allocated GPUs are used efficiently.

Quick Start

Use the sue-parallelism-discussion skill to create a validated, auditable parallelism plan for your DeepResearch scale-up experiment after launcher scripts are written and before submitting any dryrun or fullrun jobs to your selected sandbox backend.

Frequently Asked Questions about sue-parallelism-discussion

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I calculate optimal max_parallel and node packing for Slurm job submissions?▼

Calculating optimal max_parallel and node packing for Slurm jobs requires a pre-submission parallelism plan enforcing GPU-occupancy-first packing, allowing partial final nodes only to maximize hardware utilization and avoid idle GPU waste across allocated nodes.

What is GPU-occupancy-first packing for HPC scale-up experiments?▼

GPU-occupancy-first packing for HPC scale-up experiments is a node packing strategy that fills complete nodes with exactly one worker per GPU, only allowing partial final nodes, to maximize hardware utilization and eliminate wasted GPU resources during wave scheduling.

How do I prevent failed job submissions caused by exceeding Slurm queue limits on LUMI or Snellius?▼

Preventing failed job submissions from Slurm queue limits on LUMI or Snellius requires validating your wave scheduling and dryrun concurrency against queue constraints before submission, ensuring your parallelism plan complies with backend-specific limits.

Does this parallelism planning workflow support RunPod and AutoDL sandbox backends?▼

Yes, this parallelism planning workflow supports RunPod and AutoDL sandbox backends, alongside LUMI, Snellius, NM5, and Brev, covering dryrun and fullrun concurrency, node-packing strategy, and wave scheduling for ML scale-up experiments.

When do I need a pre-submission parallelism gate for ML experiment runs?▼

You need a pre-submission parallelism gate for ML experiment runs after writing launcher scripts and before submitting any dryrun or fullrun jobs, acting as a mandatory deliberation step to ensure all parallelism plans are explicit and auditable.

What are the limitations of unplanned parallelism for DeepResearch scale-up experiments?▼

Unplanned parallelism for DeepResearch scale-up experiments causes wasted GPU resources, failed job submissions, and inconsistent experiment results, failing to meet debate-gate validation and consistency-circle audit requirements required by SUE family contracts.

Related Skills

Explore More Education & Research Skills→
caiokfcaiokf

gob-background-jobs

Manage parallel background commands with gob job IDs and output capture.

Community
Advanced
ovachieverovachiever
52

tensorboard

Visualize ML training metrics and debugging signals via interactive dashboards.

Community
Intermediate
mhattingpetemhattingpete
657

timeline-creator

Create interactive HTML timelines and Gantt-chart roadmaps from project data.

Community
Basic
matsunagalabmatsunagalab
11

HPC Run

Submit and manage SLURM-based MDClaw workloads on HPC clusters.

Official
Advanced