Add PyTorch FSDP2 sharding and distributed checkpointing to training scripts.
npx skills add https://github.com/ricable/mcai --skill pytorch-fsdp2-ricable
Please help me install this Agent Skill. Skill: pytorch-fsdp2 Source: https://github.com/ricable/mcai/tree/main/.agents/skills/pytorch-fsdp2 Command: npx skills add https://github.com/ricable/mcai --skill pytorch-fsdp2-ricable
This Skill helps you upgrade an existing PyTorch training loop to use FSDP2 when model parameters, gradients, or optimizer state no longer fit on one GPU.
Use this skill to retrofit a PyTorch training script with FSDP2 sharding, DTensor-safe optimization, and distributed checkpointing.
High-intent search queries and answers about installing and using this skill.
Prevent OOM errors by retrofitting your PyTorch training script with FSDP2 bottom-up module sharding. This splits parameters, gradients, and optimizer state across GPUs using DeviceMesh-based parallelism for memory-heavy networks.
You need distributed checkpointing with DCP when training large models across multiple GPUs and require resumable checkpoints. It enables checkpoint portability and safe save/load operations for sharded DTensor parameters.
Yes, you can configure mixed precision and CPU offload with FSDP2 sharding. The Skill applies these memory optimizations alongside DeviceMesh composition and bottom-up module sharding for large distributed transformer training.
DTensor parameter handling requires optimizer construction after sharding so updates target the correct parameters. This Skill ensures DTensor-aware optimizer setup following bottom-up module sharding to maintain correct distributed updates.
FSDP2 application requires existing PyTorch training loops to support DTensor parameter handling and hook-safe model invocation. It targets distributed transformer and large-model workflows needing bottom-up module sharding and DeviceMesh-based parallelism.
handsomelong922Integrate PyTorch FSDP2 into training scripts for per-parameter sharding.
arsityShard model parameters across devices with PyTorch FSDP2 for large-model training.
MesferAliImplement PyTorch FSDP2 distributed training with DTensor sharding and checkpointing.
ovachieverApply PyTorch FSDP for sharded, mixed-precision distributed training of large models.
