LMMs-Lab
Official@evolvinglmms-lab · Singapore
Feeling and building multimodal intelligence.
Agent Skills by LMMs-Lab
Showing 9 vetted skills indexed across 2 GitHub repositories.
EvolvingLMMs-Labcu-lengths-attention-flow
Analyze cu_lengths attention boundaries across ViT and LLM stages in LLaVA-OneVision2.
EvolvingLMMs-Labdistributed-offline-packing
Pack SFT JSONL samples into Energon WebDataset shards across multiple nodes.
EvolvingLMMs-Labmegatron-checkpoint-layout
Diagnose and convert Megatron checkpoint shard layouts across TP, PP, and EP dimensions.
EvolvingLMMs-Labllava-onevision2-consistency
Compare HuggingFace and Megatron/MCore checkpoints for LLaVA-OneVision2 behavioral consistency.
EvolvingLMMs-Lablength-pool-sort-dataset
Explain cross-rank length synchronization in LengthPoolSortDataset for multi-GPU training.
EvolvingLMMs-Labmerge-ov2
Merge standalone ViT encoder and LLM into a unified LlavaOnevision2 HuggingFace checkpoint.
EvolvingLMMs-Laboffline-packing-env-vars
Configure OFFLINE_PACKING_BMR and OFFLINE_PACKED_DATA for LLaVA-OneVision2 training.
EvolvingLMMs-Labcommit-message
Enforce Conventional Commits formatting with type: subject and body guidelines.
EvolvingLMMs-Lablmms-eval-guide
Guide AI coding agents through the lmms-eval codebase for LMM evaluation.
Frequently Asked Questions About LMMs-Lab
FAQPage SchemaWhat specific tasks does LMMs-Lab enable for model engineers?▼
Engineers use these capabilities to diagnose Megatron checkpoint layouts, merge standalone ViT and language components into unified checkpoints, and synchronize multi-GPU training data. It provides the technical framework for managing LLaVA-OneVision2 behavioral consistency across different training backends and distributed node configurations.
Which technical personas benefit from these engineering protocols?▼
These protocols are designed for machine learning infrastructure engineers and research scientists focused on large-scale multimodal model training. It specifically targets those managing distributed training clusters, checkpoint conversion pipelines, and high-performance data ingestion for complex vision-language architectures.
What are the prerequisites for implementing these training protocols?▼
Implementation requires an existing LLaVA-OneVision2 environment, access to Megatron-Core or HuggingFace model architectures, and a distributed compute cluster. Users must also configure specific environment variables like OFFLINE_PACKING_BMR to ensure correct shard alignment during the dataset packing and training phases.