A backend engineer's journey into AI, ML, and LLMs.

A study log built in the open. Each note is a step in actually understanding a thing — not bookmarking it.

The forward pass — graph and matrix, side by side

An 8-step interactive walkthrough of an MLP forward pass. Watch the graph view light up step-by-step alongside the matrix view — the same computation in two languages.

Open the viz →

Prereqs

Linear algebra, calculus, tensors — only the parts I keep needing. 3 notes

Linear algebra

Vectors, matrices, dot products, broadcasting — the machinery every ML lib runs on.

In progress ↗

Calculus

Derivatives, gradients, chain rule — just enough to derive backprop on paper.

In progress ↗

Tensors

Shapes, axes, einsum, broadcasting — speaking PyTorch's native dialect.

In progress ↗

Foundations

The shared mental model the rest of the repo builds on. 6 notes

Forward pass — interactive Visualization

8-step walkthrough of an MLP forward pass — graph and matrix views, side by side.

Live ↗

Hyperparameters Notes

What hyperparameters are, how people choose them, and the short list that actually matters.

Live ↗

Backpropagation

Reverse-mode autodiff. The gradient that makes training possible — built up from chain rule.

In progress ↗

Training loop

Forward → loss → backward → step. The four lines that train every neural network.

In progress ↗

Transformers

Self-attention, multi-head, position encodings — the architecture under every modern LLM.

In progress ↗

PyTorch

nn.Module, autograd, optimizers — the working vocabulary, not a tutorial.

In progress ↗

Karpathy · Zero to Hero

Walkthroughs of the canonical from-scratch series. 3 notes

micrograd

Autograd in ~150 lines. Build the gradient engine before trusting one.

In progress ↗

makemore

From bigrams to a tiny neural language model — the on-ramp before transformers.

In progress ↗

GPT from scratch

Build a small GPT end-to-end, training included. The current focus.

Current focus ↗