perf-sequence-packing

Sequence packing improves large language model training efficiency by configuring PackedSequenceSpecs to consolidate sequences, optimizing memory usage and extending context length during fine-tuning.

Offline packed SFT is a large language model fine-tuning technique that packs sequences offline to ensure stable training with extended context lengths. You need it when adjusting sequence lengths for longer context training.

Yes, in-batch packing is supported for vision-language model fine-tuning to facilitate improved training throughput. It adjusts packing parameters to optimize memory usage during production environment training.

To configure PackedSequenceSpecs for long-context training, set sequence length and packing specifications in the training setup, then run the training process with these parameters to handle long context sequences efficiently.

Yes, sequence packing works for both large language models and vision-language models. It facilitates offline packed SFT for LLMs and in-batch packing for vision-language models to improve training throughput and memory efficiency.