train-debug

Diagnose ML training issues from symptom to root cause across GPU setups.

2|Updated Mar 20, 2026
One-click install
npx skills add https://github.com/Gonglitian/agent-skills --skill train-debug
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: train-debug
Source: https://github.com/Gonglitian/agent-skills/tree/main/skills/train-debug
Command: npx skills add https://github.com/Gonglitian/agent-skills --skill train-debug

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

ML training often encounters crashes, non-converging losses, NaN/Inf gradients, OOM, and slowdowns that block progress. This skill provides a structured diagnostic workflow to identify root causes and guide efficient fixes.

Core Features & Use Cases

  • Structured triage workflow that maps symptoms to root causes and recommended actions.
  • Category-specific diagnosis with concrete fixes, verification steps, and best practices for multi-GPU training.
  • Clear guidance for producing reproducible results and validating fixes in production-like pipelines.

Quick Start

Describe the observed issue, run triage steps to identify root cause, and apply a safe, incremental fix while verifying results.

Frequently Asked Questions about train-debug

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I debug ML training crashes and NaN gradients?

Debug ML training crashes and NaN gradients by following a structured triage workflow that checks logs, assesses GPU state, and maps symptoms to root causes for safe, incremental fixes. This ensures minimal changes while verifying results.

What is the best way to diagnose out of memory errors in multi-GPU training?

Diagnose out of memory errors in multi-GPU training by assessing GPU state and applying category-specific diagnosis steps. This workflow identifies root causes for memory errors and provides concrete fixes with verification steps to restore training.

Why does my ML training slow down across multiple GPUs?

ML training slowdowns across multiple GPUs are diagnosed through a structured workflow that checks GPU state and logs to identify bottlenecks. This triage process maps the performance symptoms to root causes and recommends safe incremental optimizations.

How do I verify ML training fixes and ensure reproducible results?

Verify ML training fixes and ensure reproducible results by applying minimal changes and validating them in production-like pipelines. The diagnostic workflow enforces step-by-step verification to confirm the resolved issue produces stable outcomes.

Can I use a diagnostic workflow to fix non-converging losses in ML training?

Yes, you can use a diagnostic workflow to fix non-converging losses in ML training. It guides you through structured triage steps from symptom to root cause, enforcing safe log checks and incremental fixes to restore convergence.