skills.rest
karpathykarpathyCommunity·1 Agent Skills Included

nanochat

Train and chat with your own GPT model cheaply

Trains a complete GPT-style language model from scratch on a single GPU node, covering tokenization, pretraining, finetuning, evaluation, and inference. Removes the complexity of hyperparameter tuning by auto-configuring everything from one depth setting. Lets you reproduce GPT-2 capability for about $48 and chat with the result over a simple command line.
57,075|7,908
↗
npx skills add karpathy/nanochat --all -g -y

All Skills in This Repository (1)

Pure Emerald Level Indicators
📦 In Repo
karpathykarpathy

read-arxiv-paper

Fetch and unpack arXiv LaTeX sources, then write Markdown summaries.

Community
Basic

Frequently Asked Questions

FAQPage Schema
How to install nanochat?â–¼

Run `npx skills add karpathy/nanochat --all -g -y` in your terminal to install all tools in this suite globally.

How much does it cost to train GPT-2 with nanochat?â–¼

About $48 on an 8xH100 GPU node for roughly 2 hours of training, and as low as $15 on spot instances.

Can nanochat run on a single GPU or CPU?â–¼

Yes. It runs on one GPU with automatic gradient accumulation, and the runcpu.sh script supports CPU or Apple Silicon for small experiments.

How do I chat with my trained model?â–¼

After training finishes, run `python -m scripts.chat_cli` to talk to your model directly in the terminal.

Do I need to tune hyperparameters in nanochat?â–¼

No. You set a single depth value and all other settings like width, learning rate, and training length are calculated automatically.

Related Repositories in Software Engineering

View All in Software Engineering→
openclawopenclaw
388.4k

openclaw

Run a personal AI assistant across your devices and chat apps

Official📦 120 Skills
obraobra
280.0k

superpowers

Gives coding agents a disciplined workflow from idea to merged code

Community📦 15 Skills
facebookfacebook
248.4k

react

AI agent skills for building, testing, and porting React core

Official📦 14 Skills