skills.rest
nishide-dev
nishide-devCOMMUNITY
Explore all skills by this author →
GitHub

ml-transformers

Build Hugging Face Transformer fine-tuning pipelines on PyTorch Lightning.

Updated Feb 6, 2026
Search Tags:#lora#huggingface#transformers#fsdp#deepspeed#pytorch-lightning#torchmetrics
One-click install
npx skills add https://github.com/nishide-dev/claude-code-ml-research --skill ml-transformers
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ml-transformers
Source: https://github.com/nishide-dev/claude-code-ml-research/tree/main/skills/ml-transformers
Command: npx skills add https://github.com/nishide-dev/claude-code-ml-research --skill ml-transformers

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

It solves the problem of integrating Hugging Face Transformers into PyTorch Lightning training workflows without breaking loss computation, metrics, or scalability.

Core Features & Use Cases

  • LightningModule + Transformers integration pattern: Encapsulates Transformer models for clean training_step/validation_step flows while relying on Transformers’ built-in label-aware loss computation.
  • Production-ready training building blocks: Covers dynamic padding with DataCollatorWithPadding, optimizer configuration with correct warmup scheduling, and checkpoint-friendly reproducibility via save_hyperparameters().
  • Scalable training and efficient fine-tuning: Guides distributed strategies (DDP/FSDP/DeepSpeed) and parameter-efficient methods (LoRA/QLoRA) with practical evaluation guidance using TorchMetrics.

Use case example: You want to fine-tune a BERT/LLM for text classification or causal language modeling while tracking train/val loss and metrics correctly across multiple GPUs, with optional FSDP/DeepSpeed scaling and LoRA adapters to reduce GPU memory needs.

Quick Start

Ask the assistant to generate a LightningModule that wraps a Hugging Face model for your task using correct forward/training_step separation, HF label-based loss, TorchMetrics for evaluation, and DataCollatorWithPadding for efficient batching.

Frequently Asked Questions about ml-transformers

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I integrate Hugging Face Transformers with PyTorch Lightning for fine-tuning?▼

You integrate Hugging Face Transformers with PyTorch Lightning by wrapping the model in a LightningModule, using the Transformers built-in label-aware loss computation to handle training_step and validation_step flows correctly.

What's the best way to scale LLM fine-tuning across multiple GPUs using FSDP or DeepSpeed?▼

The best way to scale LLM fine-tuning across multiple GPUs is to apply distributed strategies like FSDP or DeepSpeed within your PyTorch Lightning pipeline, ensuring reliable and scalable NLP training without breaking loss or metric computations.

Can I use LoRA or QLoRA for parameter-efficient fine-tuning in a Lightning training pipeline?▼

Yes, you can use LoRA or QLoRA for parameter-efficient fine-tuning within a Lightning training pipeline to reduce GPU memory needs while fine-tuning BERT or LLMs for text classification and causal language modeling tasks.

How do I handle dynamic padding and tokenization when building a Transformer fine-tuning pipeline?▼

You handle dynamic padding and tokenization by using Hugging Face DataCollatorWithPadding for efficient batching, alongside configuring optimizers with correct warmup scheduling and managing robust metric evaluation with TorchMetrics.

Does PyTorch Lightning work with Hugging Face datasets for tracking train and validation metrics?▼

Yes, PyTorch Lightning works with Hugging Face datasets by integrating TorchMetrics for robust evaluation, allowing you to correctly track train and validation loss and metrics across multiple GPUs during Transformer fine-tuning.

Related Skills

Explore More Education & Research Skills→
ChimeraFoundationaChimeraFoundationa

pytorch-lightning

Abstract PyTorch training boilerplate into LightningModule and Trainer workflows.

Community
Advanced
ovachieverovachiever
52

transformers

Load, infer, and fine-tune pre-trained transformer models across modalities.

Community
Advanced
brittaniebuffiecsubrittaniebuffiecsu
1

pytorch-lightning

Train PyTorch models with high-level APIs and distributed strategies.

Community
Advanced
Estrella-231Estrella-231

transformers

Run inference and fine-tuning with Hugging Face Transformers models.

Community
Intermediate