alpamayo1-x-rl

You can automate RL post-training for Alpamayo VLM action models by using Cosmos-RL and GRPO to convert checkpoints, curate a PAI mini subset, launch training, and export the final policy checkpoint to a HuggingFace directory.

Converting an Alpamayo release checkpoint to a training-ready format involves transforming the model weights so they can be used to launch policy and rollout replicas via Cosmos-RL for subsequent GRPO reinforcement learning.

Yes, you need flash-attn and cosmos-rl along with alpamayo_r1 and alpamayo-recipes to run GRPO training on the PAI dataset and manage the policy and rollout replicas.

GRPO reinforcement learning with VLM action models on the PAI dataset works by launching policy and rollout replicas via Cosmos-RL to train the agent and then exporting the resulting policy checkpoint back to a HuggingFace directory.

Yes, you can export the trained policy checkpoint back to a HuggingFace directory after RL training, converting the final model weights from the Cosmos-RL environment for standard inference use.

A key limitation is that using Cosmos-RL for Alpamayo post-training requires specific dependencies including alpamayo_r1, alpamayo-recipes, and flash-attn, restricting usage to environments that support these frameworks.