mlc-aiOfficial·8 Agent Skills Included

pith-train

MoE model training, profiling, memory estimation, and validation

Trains Mixture-of-Experts language models with overlapped pipeline parallelism, FP8 kernels, and 4D distributed scaling. Captures and analyzes GPU profiles, estimates peak memory before runs, and compares loss curves to catch regressions. Eliminates manual SLURM setup, checkpoint conversion, and experiment tracking so training workflows run faster with fewer errors.

327|31

↗

npx skills add mlc-ai/pith-train --all -g -y

Available:

Gives your AI agent the project architecture, setup commands, testing workflows, and training launch procedures it needs to work on this MoE training framework correctly.

All Skills in This Repository (8)

Pure Emerald Level Indicators

📦 In Repo

mlc-ai

analyze-nsys-profile

Query Nsight Systems profiles to measure compute/communication overlap and inspect per-rank streams.

capture-nsys-profile

Capture Nsight Systems traces for PithTrain runs across parallelism configurations.

add-memory-prints

Add memory profiling instrumentation to distributed MoE training workflows.

setup-benchmark-inputs

Prepare tokenized corpus shards and HF-DCP checkpoints for MoE benchmarks.

estimate-memory

Estimate peak GPU memory usage for DualPipeV MoE training runs.

add-new-model

Create MoE model skeletons for PithTrain and wire them into training and inference workflows.

validate-correctness

Run paired base and feature branch training validations and diff per-step metrics.

launch-with-slurm

Dispatch SLURM workloads by wrapping srun inside the current allocation.

Official

Advanced

Frequently Asked Questions

FAQPage Schema

How to install PithTrain?▼

Run `npx skills add mlc-ai/pith-train --all -g -y` in your terminal to install all skills in this suite globally.

How to estimate GPU memory before training?▼

The estimate-memory skill simulates the full training schedule and reports peak memory, per-component breakdown, and whether your config fits on your GPUs.

How to profile MoE training performance?▼

The capture-nsys-profile and analyze-nsys-profile skills record a GPU trace and measure compute-communication overlap per pipeline stage to find bottlenecks.

Can I add a new model to PithTrain?▼

Yes. The add-new-model skill walks through a gated, phase-by-phase workflow covering the model file, framework wiring, correctness tests, and checkpoint conversion.

How to check if code changes break training?▼

The validate-correctness skill runs identical short trainings on both branches and diffs per-step loss curves, failing if divergence exceeds tolerance.