pytorch-lightning

Organize PyTorch code into LightningModules and configure Trainers for distributed deep learning.

Updated Oct 7, 2022
One-click install
npx skills add https://github.com/tamagusko/linux-cfg --skill pytorch-lightning-tamagusko
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: pytorch-lightning
Source: https://github.com/tamagusko/linux-cfg/tree/main/dotfiles/claude/skills/pytorch-lightning
Command: npx skills add https://github.com/tamagusko/linux-cfg --skill pytorch-lightning-tamagusko

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires lightning, torch, and includes scripts (resource) and references (resource) components.

What problem does it solve? Raw PyTorch training code mixes research logic with engineering boilerplate like device management, distributed synchronization, and checkpointing, making experiments hard to reproduce and scale across GPUs. ## Core Features & Use Cases - LightningModule Templates: Structure models into training, validation, test, and optimizer configuration steps with automatic metric logging and hyperparameter saving. - Trainer & Distributed Strategies: Configure multi-GPU and multi-node training with DDP, FSDP, or DeepSpeed, plus mixed precision, gradient accumulation, and checkpointing. - DataModules, Callbacks & Logging: Build reusable data pipelines, extend training with callbacks like EarlyStopping and ModelCheckpoint, and log to TensorBoard, W&B, MLflow, or CSV. - Use Case: You need to train a transformer across 8 GPUs with limited memory. Use this Skill to wrap your model in a LightningModule, select the FSDP strategy with bf16 mixed precision, and add checkpointing and early stopping callbacks. ## Quick Start Help me convert my PyTorch training script into a PyTorch Lightning LightningModule with a Trainer configured for multi-GPU DDP training.

Frequently Asked Questions about pytorch-lightning

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I convert a PyTorch training loop to PyTorch Lightning?

Move your model into a LightningModule subclass and implement training_step, validation_step, and configure_optimizers. The Trainer then handles the loop, backward pass, device placement, and logging automatically.

DDP vs FSDP vs DeepSpeed: which distributed strategy should I use?

Use DDP for models under roughly 500M parameters that fit in GPU memory. Use FSDP for models of 500M+ parameters, and DeepSpeed when you need CPU or disk offloading and fine-grained ZeRO configuration.

Does PyTorch Lightning support logging to Weights & Biases and TensorBoard?

Yes, Lightning includes built-in loggers for TensorBoard, W&B, MLflow, Neptune, Comet, and CSV. Pass one or more logger instances to the Trainer and record metrics with self.log inside your LightningModule.

Why are my validation metrics wrong when training on multiple GPUs?

Metrics logged without synchronization are averaged only within each process. Pass sync_dist=True to self.log so Lightning aggregates the metric correctly across all devices in distributed training.

How do I resume PyTorch Lightning training from a checkpoint?

Enable the ModelCheckpoint callback with save_last=True, then call trainer.fit with ckpt_path pointing to the checkpoint file. Model hyperparameters saved via save_hyperparameters are restored automatically.