skills.rest
sayalinvidia
sayalinvidiaCOMMUNITY
Explore all skills by this author →
GitHub

nemo-mbridge-perf-sequence-packing

Configure and validate packed sequences and long-context training in Megatron-Bridge.

Updated Apr 16, 2026
Search Tags:#finetuning#long-context#vlm#sequence-packing#packed-sequences#context-parallel#nemo-mbridge
One-click install
npx skills add https://github.com/sayalinvidia/sayali-skills-test --skill nemo-mbridge-perf-sequence-packing
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: nemo-mbridge-perf-sequence-packing
Source: https://github.com/sayalinvidia/sayali-skills-test/tree/main/skills/nemo-mbridge-perf-sequence-packing
Command: npx skills add https://github.com/sayalinvidia/sayali-skills-test --skill nemo-mbridge-perf-sequence-packing

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill guides configuring and validating packed sequence paths and long-context training in Megatron-Bridge, distinguishing offline packed SFT for LLMs from in-batch packing for VLMs, and applying the correct CP constraints.

Core Features & Use Cases

  • Offline packed SFT configuration via PackedSequenceSpecs to support long sequences during finetuning.
  • In-batch packing for VLM finetuning and how to switch between packing modes.
  • Context-parallelism padding rules, CUDA-graph metadata requirements, and finetuning prerequisites handling.

Quick Start

Provide a Megatron-Bridge training config that enables offline packed SFT with appropriate PackedSequenceSpecs and CP settings.

Frequently Asked Questions about nemo-mbridge-perf-sequence-packing

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I configure offline packed sequences for long-context finetuning in Megatron-Bridge?▼

You can configure offline packed sequences for long-context finetuning in Megatron-Bridge by providing a training config that enables offline packed SFT with appropriate PackedSequenceSpecs and context-parallelism settings. This validates padding rules and CUDA-graph metadata prerequisites.

What is the difference between offline packed SFT and in-batch packing for VLMs?▼

Offline packed SFT for LLMs and in-batch packing for VLMs are distinct sequence packing paths in Megatron-Bridge. This skill specifies how to distinguish between them and switch packing modes correctly during VLM finetuning workflows.

How do I apply context-parallelism padding rules during sequence packing?▼

Context-parallelism padding rules are applied during sequence packing by configuring PackedSequenceSpecs within your Megatron-Bridge training pipeline. This ensures correct CP constraints and CUDA-graph metadata requirements are met for packed sequences.

Do I need specific finetuning prerequisites to use packed sequences in Megatron-Bridge?▼

Yes, using packed sequences in Megatron-Bridge requires specific finetuning prerequisites, including correct PackedSequenceSpecs configuration and validation of CUDA-graph metadata requirements. This skill guides handling these prerequisites for both LLM and VLM workflows.

Does Megatron-Bridge support in-batch packing for vision-language model finetuning?▼

Yes, Megatron-Bridge supports in-batch packing for vision-language model finetuning. This skill guides switching between offline packed SFT and in-batch packing modes while applying the correct context-parallelism padding constraints.

Related Skills

Explore More Software Engineering Skills→
NVIDIANVIDIA
2.8k

sequence-packing

Manages PackedSequenceSpecs and context-parallel constraints for Megatron-Bridge long-context training.

Official
Advanced
yo-stevenyo-steven

perf-sequence-packing

Validate and configure packed-sequence and long-context training in Megatron-Bridge.

Community
Advanced
sayalinvidiasayalinvidia

nemo-mbridge-perf-hierarchical-context-parallel

Configure hierarchical context parallelism for Megatron-Bridge training jobs.

Community
Advanced
sovr610sovr610

Data Loader Throughput + Sequence Packing

Measure and optimize dataloader throughput and sequence packing for large-scale training.

Community
Advanced