ai-learnings

Transformers

The architecture behind every modern LLM: attention, multi-head attention, the transformer block, positional encodings, layer norm, residual connections.

What’s here

• (notes added as I work through Karpathy’s GPT-from-scratch and the original paper)

Status

[~] In progress — going through alongside courses/karpathy/gpt-from-scratch/.

Prerequisites

foundations/backprop/ — you should be comfortable with how gradients flow before you stack 12 attention layers.

Resources

• Attention Is All You Need — original paper. Notes: papers/notes/attention-is-all-you-need.md.
• Jay Alammar — The Illustrated Transformer.
• Karpathy — Let’s build GPT lecture.