nanoGPT: Karpathy's minimalist GPT implementation, from Shakespeare to GPT-2
Train a character-level Shakespeare model on CPU in minutes, reproduce GPT-2 (124M) on OpenWebText, fine-tune from OpenAI checkpoints, and train on custom datasets — the hackable reference for learning transformers
- What
- Train a character-level Shakespeare model on CPU in minutes, reproduce GPT-2 (124M) on OpenWebText, fine-tune from OpenAI checkpoints, and train on custom datasets — the hackable reference for learning transformers
- Cost
- Free
- Needs
- Python with PyTorch for real training runs; CPU suffices for the Shakespeare workflow, GPUs needed for GPT-2 scale
- Install
- Copy the installer prompt below into your Muse — your agent does the rest.
Curated by Skill Harbor — @orchestra-research's nanoGPT skill: the agent's guide to Karpathy's educational GPT implementation — four concrete workflows: character-level Shakespeare training in ~5 minutes on CPU (prepare → train → sample, 6 layers, 384-dim embeddings), full GPT-2 (124M) reproduction on OpenWebText with multi-GPU DDP (~4 days on 8x A100), fine-tuning from OpenAI GPT-2 checkpoints on custom text, and a custom-dataset pipeline (character tokenizer → train.bin/val.bin). It documents the model config knobs, the training-speed levers (`compile=True`, bf16), common failure fixes (OOM → shrink batch/block size; slow → compile + mixed precision; poor generations → train longer, lower temperature; can't load GPT-2 weights → check `init_from` names), hardware requirements per workflow, and when to graduate to alternatives (HuggingFace Transformers for production, Megatron-LM for large-scale, LitGPT for production architectures). Honest caveats: an educational reference, GPT-2 (124M)-scale only — not a production training framework; OpenWebText preparation takes ~1 hour and the 124M run needs serious GPU time; benchmark claims are the skill's. MIT licensed. Skill Harbor never reviews the code, review it yourself before use. Discovered via skills.sh.
Version:
Install
Prerequisites: Python with PyTorch for real training runs; CPU suffices for the Shakespeare workflow, GPUs needed for GPT-2 scale Install "nanoGPT: Karpathy's minimalist GPT implementation, from Shakespeare to GPT-2" for me. It gives my agent @orchestra-research's nanoGPT playbook: four workflows (CPU Shakespeare in minutes, full GPT-2 124M reproduction on OpenWebText, fine-tuning from OpenAI checkpoints, custom datasets), config knobs, speed levers (torch.compile, mixed precision), common failure fixes, hardware requirements per workflow, and when to graduate to production alternatives. MIT licensed. Repository: https://github.com/orchestra-research/ai-research-skills/blob/main/01-model-architecture/nanogpt/SKILL.md 1. Fetch the SKILL.md file (and any helper files) from the repository path into a temporary folder and summarize what it does in one or two sentences. 2. Safety check: review the SKILL.md and scripts for anything suspicious (unexpected network calls, shell commands, credential harvesting). This repo should contain zero secrets in code, credentials only via the secure vault, allowed hosts declared in the SKILL.md. Verify that holds here; STOP on any red flag and tell me. 3. Install it as a skill: copy SKILL.md and its helper files into the agent's skills directory, in a folder named "nanogpt". 4. Verify with no network calls: frontmatter valid, files in place. 5. Report what was installed, where, and what I still need to do myself (e.g. tell the agent which workflow to run — Shakespeare-on-CPU quick start, GPT-2 reproduction, checkpoint fine-tuning, or a custom dataset — and confirm the hardware available; OpenWebText prep takes ~1 hour, plan GPU time for 124M). GitHub is optional: if I have a GitHub account or the gh CLI, you may use it; otherwise public access is fine. Never require it unless it's in the prerequisites above. Rules: don't touch anything outside the temp folder and the install target. If anything looks off, stop and ask me.
Questions
How do I install a build?
Every product page includes a copy-paste install prompt. Paste it into your Muse and it sets the build up for you — no manual configuration.
Where does my money go?
Straight to the seller. Skill Harbor never processes payments: checkout happens on the seller’s own page, usually Stripe.
What does the ✓ next to a creator’s name mean?
It means we confirmed the identity of the person behind the listing. It says nothing about the code itself — always check a build before installing it.