skills.rest
mlc-ai
mlc-aiOFFICIAL
Explore all skills by this author →
GitHub

validate-correctness

Run paired base and feature branch training validations and diff per-step metrics.

327|31|Updated Mar 21, 2026
Search Tags:#workflow#training#validate#correctness#regression-testing#moe#log-compare
One-click install
npx skills add https://github.com/mlc-ai/pith-train --skill validate-correctness
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: validate-correctness
Source: https://github.com/mlc-ai/pith-train/tree/main/.agents/skills/validate-correctness
Command: npx skills add https://github.com/mlc-ai/pith-train --skill validate-correctness

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

Validates that code changes do not break training correctness by comparing per-step loss curves between a base branch and the current feature branch. Use when user asks to "validate correctness", "check if changes break training", "compare loss curves", "run a regression test", or "verify my changes are correct". The user specifies which model to validate and at which parallelism mesh (PP/EP/CP) — do not infer this from git diff.

Core Features & Use Cases

  • Runs a paired-base/feature training validation to surface regressions in cross-entropy and load-balance metrics.
  • Diffs per-step metrics and reports pass/fail with a defined tolerance; supports adjustable context and sequence length.
  • Use Case: QA for model changes before merging feature branches and validating MOE mesh configurations.

Quick Start

Run the validation with your target model and mesh to detect regressions.

Frequently Asked Questions about validate-correctness

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I validate training correctness and check if code changes break my model?▼

To validate training correctness, you compare per-step loss curves between a base branch and a feature branch. This regression test surfaces training regressions by diffing cross-entropy and load-balance metrics against enforced tolerances to report PASS or FAIL.

How do I run a regression test comparing loss curves across different MOE meshes?▼

Specify your target model and parallelism mesh (PP/EP/CP) to run paired-base and feature validations. The process diffs per-step cross-entropy and load-balance metrics, enforcing tolerance checks to report a pass or fail outcome with detailed logs.

Can I configure context length and batch size when validating model training regressions?▼

You can configure steps, contexts, sequence lengths, and batch sizes for your validations. These adjustable parameters ensure the cross-entropy and load-balance metric comparisons fit your specific target model and parallelism mesh configuration.

What metrics are compared when checking for training regressions in MOE models?▼

Cross-entropy and load-balance metrics are compared when checking for training regressions. The validation diffs these per-step metrics between base and feature branches, applying tolerance checks to report a pass or fail status with detailed logs.

Do I need to manually specify the parallelism mesh for model validation tasks?▼

You must explicitly specify the target model and parallelism mesh (PP/EP/CP) for validation. The configuration is never inferred from git diffs, ensuring the paired-base and feature branch metrics are accurately compared.

Related Skills

Explore More Software Engineering Skills→
sayalinvidiasayalinvidia

nemo-mbridge-perf-parallelism-strategies

Select and tune Megatron Bridge parallelism configurations for model and hardware.

Community
Advanced
CDO-07-NewCDO-07-New

eval-harness

Define repeatable evaluation harnesses for Claude Code task outcomes.

Official
Intermediate
tadod12tadod12

moe-training

Optimize Mixture of Experts training with DeepSpeed or HuggingFace.

Community
Advanced
skywalkjianskywalkjian

ai-experiment-report

Generate structured Markdown experiment reports from training scripts and logs.

Community
Advanced