What problem does it solve? Measuring large language model pretraining performance on NVIDIA GPUs requires coordinating container setup, Megatron data preprocessing, distributed training execution, and log-based metric extraction. This Skill automates that entire evaluation workflow for Qwen3-8B so engineers get comparable, reproducible throughput numbers. ## Core Features & Use Cases - Automated Evaluation Pipeline: Runs container startup, Megatron data preprocessing, NeMo-based pretraining, and metric collection through three dedicated scripts (preprocess.sh, pretrain.sh, calc.sh). - Standardized Throughput Metrics: Extracts tokens_per_sec_per_gpu from training logs, trims warmup and tail steps, and computes per-GPU and total throughput written to result.json. - Use Case: A hardware evaluation engineer needs to verify Qwen3-8B pretraining performance on an 8-GPU NVIDIA node. The Skill launches the NeMo container, preprocesses the arxiv sample dataset, runs 100 training steps, and produces a structured result.json with average tokens per second per GPU. ## Quick Start Ask the agent to evaluate Qwen3-8B pretraining performance on NVIDIA GPUs and collect the tokens_per_sec_per_gpu metric.
