Automate Alpamayo-1.5 VLM fine-tuning on PAI or LingoQA datasets.
npx skills add https://github.com/NVlabs/alpamayo-recipes --skill alpamayo1-5-sft
Please help me install this Agent Skill. Skill: alpamayo1-5-sft Source: https://github.com/NVlabs/alpamayo-recipes/tree/main/recipes/alpamayo1_5_sft Command: npx skills add https://github.com/NVlabs/alpamayo-recipes --skill alpamayo1-5-sft
This Skill automates the end-to-end process of fine-tuning the Alpamayo-1.5 VLM action model on PAI (with navigation conditioning) or LingoQA (for visual question answering), reducing the time and complexity of training and fine-tuning custom agents.
Run the 'alpamayo1-5-sft' skill to fine-tune the Alpamayo-1.5 model on PAI for navigation conditioning. Provide the necessary paths to the PAI dataset, navigation annotations, and Alpamayo-1.5 checkpoint.
High-intent search queries and answers about installing and using this skill.
Fine-tune a VLM action model for autonomous vehicle navigation by running the Alpamayo-1.5 pipeline on the PAI dataset with navigation conditioning to predict trajectories guided by route instructions. The process automates training and evaluation end-to-end.
Yes, you can train a visual question answering model for driving scenes using the LingoQA dataset. The Alpamayo-1.5 fine-tuning pipeline automates the training process specifically for driving scene question answering.
The datasets supported for Alpamayo-1.5 supervised fine-tuning are PAI and LingoQA. PAI is used for navigation conditioning and trajectory prediction, while LingoQA is used for visual question answering in driving scenes.
DeepSpeed is supported to ensure efficient training during the two-stage pipeline, though the optional Stage 2 specifically handles trajectory diffusion expert training. The pipeline also leverages gradient checkpointing to optimize resources.
The Alpamayo-1.5 model requires converting the release checkpoint to A1 format before training begins. This conversion is a necessary preparatory step before executing the two-stage training pipeline on your prepared datasets.
NVlabsAutomate RL post-training for Alpamayo VLM action models with Cosmos-RL and GRPO.
aayushsoamAnswer questions about images with detailed textual responses.
HermeticOrmusImplement distributed training patterns with PyTorch DDP, FSDP, and DeepSpeed ZeRO-3.
ovachieverAutomate LLaMA-family model fine-tuning via the Llama-Factory WebUI.
