# GPT From Scratch

A working transformer-based language model, built end-to-end from Karpathy's *Let's build GPT* lecture.

## Status

[~] In progress.

## Prerequisites

- [makemore](../makemore/) helps but isn't strictly required.
- [`foundations/transformers/`](../../foundations/transformers/) — at least a sense of what attention is.

## Resources

- Karpathy — *Let's build GPT: from scratch, in code, spelled out*.
- [`papers/notes/attention-is-all-you-need.md`](../../papers/notes/attention-is-all-you-need.md).