sequence-packing

What problem does it solve?

This Skill helps users configure and validate packed sequence workflows in Megatron-Bridge, enabling stable long-context training by distinguishing offline packed SFT paths for LLMs from in-batch packing paths for VLMs, and applying the correct CP constraints.

Core Features & Use Cases

• Distinguish two packing paths: offline packed SFT for LLM finetuning and in-batch packing for VLM finetuning, with aligned CP settings.
• Provide configuration anchors and validation guidance to ensure pad lengths, sequence lengths, and mult requirements are respected during training.
• Typical use case includes planning or debugging Megatron-Bridge training runs that require long-context sequences and context-parallelism.

Quick Start

Configure Megatron-Bridge to enable offline packed SFT for LLM finetuning or in-batch packing for VLMs, and apply the correct CP constraints.