nanogpt-training
CommunitySingle-GPU GPT-2 scale training.
AuthorKaiserWhoLearns
Version1.0.0
Installs0
System Documentation
What problem does it solve?
Train GPT-2 scale models efficiently on a single GPU, enabling rapid prototyping without heavy hardware.
Core Features & Use Cases
- Efficient single-GPU pretraining for GPT-2-scale architectures (e.g., ~124M parameters).
- Tokenized data loading from HuggingFace Hub or local shards with memory-friendly handling.
- Mixed-precision training and configurable optimizers (AdamW, Muon) with a flexible training loop for experimentation.
Quick Start
Run the included minimal training script to reproduce a GPT-2-scale training workflow on a single GPU.
Dependency Matrix
Required Modules
None requiredComponents
references
💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: nanogpt-training Download link: https://github.com/KaiserWhoLearns/skillsbench/archive/main.zip#nanogpt-training Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 471,000+ vetted skills library on demand.