sue-fullrun-diagnose

What problem does it solve?

This skill resolves the problem of unquantified GPU hour waste and missing GPU resource utilization metrics after a completed ML/HPC scale-up experiment full run, which occurs when initial run summaries have MISSING GPU-R cells or users suspect low GPU efficiency without actionable breakdowns.

Core Features & Use Cases

• Post-run GPU waste attribution: Reduces per-shard timing and log artifacts to assign GPU hours to discrete waste buckets (cancelled retries, setup overhead, compile tax, etc.) without launching new compute.
• Enriched diagnostic outputs: Generates per-variant stage timing CSVs, error summaries, and updated summary reports that replace MISSING metrics with real values.
• Use Case: After a DINOv3 evaluator fullrun on LUMI shows 15-25% GPU efficiency with missing useful_gpu_work_hours values, use this skill to identify that 40% of waste is pre-stage tail overhead and 17% is cancelled retries, so you can prioritize targeted fixes for the next run.

Quick Start

Use the sue-fullrun-diagnose skill to analyze your most recent completed fullrun bundle and fill all missing GPU efficiency metrics in the existing experiment summary.