ml-transformers

What problem does it solve?

It solves the problem of integrating Hugging Face Transformers into PyTorch Lightning training workflows without breaking loss computation, metrics, or scalability.

Core Features & Use Cases

• LightningModule + Transformers integration pattern: Encapsulates Transformer models for clean training_step/validation_step flows while relying on Transformers’ built-in label-aware loss computation.
• Production-ready training building blocks: Covers dynamic padding with DataCollatorWithPadding, optimizer configuration with correct warmup scheduling, and checkpoint-friendly reproducibility via save_hyperparameters().
• Scalable training and efficient fine-tuning: Guides distributed strategies (DDP/FSDP/DeepSpeed) and parameter-efficient methods (LoRA/QLoRA) with practical evaluation guidance using TorchMetrics.

Use case example: You want to fine-tune a BERT/LLM for text classification or causal language modeling while tracking train/val loss and metrics correctly across multiple GPUs, with optional FSDP/DeepSpeed scaling and LoRA adapters to reduce GPU memory needs.

Quick Start

Ask the assistant to generate a LightningModule that wraps a Hugging Face model for your task using correct forward/training_step separation, HF label-based loss, TorchMetrics for evaluation, and DataCollatorWithPadding for efficient batching.