ai-learnings

GPT From Scratch

A working transformer-based language model, built end-to-end from Karpathy’s Let’s build GPT lecture.

Status

[~] In progress.

Prerequisites

• makemore helps but isn’t strictly required.
• foundations/transformers/ — at least a sense of what attention is.

Resources

• Karpathy — Let’s build GPT: from scratch, in code, spelled out.
• papers/notes/attention-is-all-you-need.md.